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ABSTRACT 

The report presents findings from a 5-year 
investigation of teaching strategies and their effects on learning 
stages in severely handicapped students* Learning records of most of 
the students were found to resemble those of nonhandicapped learners, 
although some students demonstrated erratic variations in performance 
due to compliance problems* In the third project year decision-rules 
were formulated to help teachers identify problems in acquisition, 
fluency building and compliance from the students 1 performance 
records* Effectiveness of teachers who used the decision-rules was 
analysed in terms of improvements in pupil performance and/or rate of 
progress* In year 4, teachers applied a revised set of decision rules 
based on questions concerning pupil data* problems with noncompliance 
in children were noted and interventions attempted. Decision-rules 
were integrated with a minimum acceleration procedure in which the 
teacher specifies a minimum rate of change* Year 5 activities 
examined the learning of the decision-rules by 81 teachers in a wide 
variety of settings with varying amounts of training and assistance. 
Adoption of the rules resulted in successful decisions and reduced 
planning time. Cost data were also examined* Four site reports are 
presented in the document's conclusion* (CL) 
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Summary af the Project 



"An Investigation af the Phases af Learning and Facilitating Instructional Events 
for the Severely/Profoundly Handicapped" was funded in 1975. The major objectives 
of this five year project were to: 

(1) Investigate the relationship between the hypothesized stages of learning— 
acquisition, fluency building, maintenance, generalization and adaptation— and 
pupil performance in order ta determine functional definitions of eoch stage; 

(2) Determine if specific instructional strategies had a higher probability af 
facilitating pupil progress in certain stages of learning than in other stages; and 

(3) Determine if and how information from the first twa objectives could be easily 
and effectively used by classroom teachers ta improve the education of their 
severely/profoundly handicapped pupils. 

During the first twa project years, the project worked with fourteen teachers, 
twenty pupils and a total of 40 different instructional programs. Pupils' performance 
was analyzed as teachers implemented a variety af instructional strategies, although 
no attempt was made by project staff to influence the strategies used by the teochers. 
Analysis af the pupils 1 performance data showed that the first two stages at learning 
could be identified, and also indicated that strategies designed ta provide information 
on how to respond (e*g., prompting) were most effective during acquisition, while 
strategies designed to motivate performance (e.g., use af reinforcers) were most 
effective during the fluency building stage. However, it was found that teachers were 
far more likely to move a pupil to an easier instructional step or change the 
consequence for correct responses when learning failed than ta change instructional 
strategies. During the first project year teachers selected intervention strategies that 
improved pupil performance 33% of the time while during the second year teachers 
were successful 41% of the time. 

Several other discoveries were made during the first twa years. It became 
apparent that most teachers collected data on the occurocy af performance, usually 
percent correct. In addition to accuracy data, project staff collected three tyres of 
time-based data to determine the temporal changes in performance. Frequency of 
behavior (rote per minute) data were collected on all skills at first. Later, duration 
data were collected on skills in which the acceleration or deceleration of duration was 
the primary consideration in the development of fluent performance (i.e., self-help 
skills such as shoe-tying are duration deceleration targets, and time on task is a 
duration acceleration target). Latency data were collected on skills in which the 
amount of time between the conclusion of the stimulus and the start of the behavior 
were the important measure of proficiency (he*, answering question). We also found it 
necessary to modify data collection techniques for program strategies that required 
the presentation of antecedents or consequences for each pupil response. The 
"Guidelines" attached ta this report include a description of data collection 
techniques. 

During the first two years, the project found that the learning records af most of 
the pupils resembled those af nonhandtcapped learners (see Figure |). Rate-af-change 
in performance was not noticeably different, although the target behaviors and the 
relative size of instructional steps were different. 
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However, the instructional performance records of sever o I pupils showed such 
erratic variations that they were impossible to classify. Performances jumped daily 
from 100% to 0%, from rates of 20 per minute to rates of I per minute (Figure I). 
There was more variation, or bounce, from day-to-day than there was change over 
time. Extensive observation of these pupils in instructional and noninstructionat 
situation showed a correlation between compliance to commends already in the pupils' 
repertoire and variability in daily instructional tasks: The less likely the pupil was to 
comply the more erratic instructional performance was. Such performance records 
were classified as "compliance problems. 11 

During the third project year, a set of decision-rules were designed to assist 
teachers in the identification of problems In acquisition, fluency building md 
compliance from the performance records of their pupils. Those rules were 
implemented by six teachers and sixteen pupils at the Lake Washington Special 
Education Center, working closely with the research stuff. 

This first version of the decision-rules consisted of a series of "pictures" of pupil 
performance data based on both occurocy and time data The teachers drew Hnes-of- 
progress (a.k.a. trends, slopes) on a chart of the data and then compared their 
"picture" with the rule pictures. Each picture was categorized as either a "problem" 
(e.g., acquisition, fluency-building, compliance, general) or as a successful learning 
pattern. Teachers could then choose fron* a list of suggested remedial strategies for 
each problem category or select an intervention of their choice. Teachers who 
implemented the first version of decision-rules made 65% successful decisions, i.e., 
decisions that improved pupil performance and/or rate of progress. Teachers were 
sometimes successful In selecting appropriate interventions even if they did not use 
one of the specifically recommended strategies. However, teachers who did us*, 
suggested strategies were generally more successful than those that used other, non- 
recommended strategies. One result was that teachers used a wider variety of 
remedial strategies than teachers in previous years. Although this may be attributed 
to a change in the sample of teachers under study, it may also have been due to the 
variety of strategies recommended by the project procedures. 

During the fourth project year, four teachers from the Seattle School District 
applied a revised set of decision-rules in programs for 22 severely and profoundly 
handicapped pupils. That set of decision-rules consisted of a series of questions 
concerning pupil data. For example, the teacher was asked to determine if progress 
was accelerating or decelerating. If it was decelerating, the teacher next determined 
if performance was highly variable, the guidelines indicated the high probabtity of a 
compliance problem and suggested a number of strategies for dealing with that 
problem. Questions were used to determine the category of pupil performance (e.g., 
successful learning pattern, acquistion problem, compliance problem, fluency-building 
problem). A revised set of recommended instructional strategies for each category 
was included with the decision-rules. In addition to drawing I ines-of-pr ogress, 
teachers were required to determine the percentage of c*r r ect responses and the 
degree of performance variability in order to answer the questions. Again, teachers 
were free to implement suggested strategies or use others of their choice. During the 
year, teachers used the suggested strategies 80% of the time, and 80% of their 
decisions resulted in improved pupil performance. 



' The use of general "learning pictures" of correct and error rates has been reported 
by O. R. Lindley (personal communication, summer, 1979). 
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As o result of project octivities, ft beccme even mote obvious that compliance 
problems constituted a major challenge ta classroom teachers. Not only was 
instructional decision-making very difficult with nancompliant students, but even 
correct placement within a curriculum was seriously hampered, since pupils often 
responded noncompllantly during initial assessments. The problems with noncomptiant 
pupils uncovered during the first three years of the project, unanticipated by previous 
research, involving between 5*15% of the subjects, posed such severe problems that 
the project was forced ta discontinue investigations into generalization and adaption, 
and to concentrate on identifying compliance problems in instruction and strategies 
that would improve compliance. We concluded that unless a pupil were compliant, any 
type of instruction would probably be ineffective, and that, therefore, compliance was 
probably a prerequisite ta instruction. 

During the second, third and fourth project years separate investigations into 
strategies for remediating compliance problems in instructional and non-instructional 
settings were conducted. Non-instructional settings constituted those periods when 
the teacher and pupil were not engaged in a pre-planned systematic instructional 
octivity. Strategies that were successful in non-instructional settings included 
manipulation of contingencies for compliant and for noncompliant responses: food plus 
praise for compliance, and a physical mandate (a "putting through" procedure) for 
noncompliance or if the pupil failed ta respond within five seconds. It was also found 
that apiying those contingencies throughout the day, whenever a compliance command 
was given, was more successful in improving levels and stability of compliance than 
applying them in a short, specially designed, compliance session. Some pupils who 
became more compliant during non-instructional situations also showed a reduction in 
variability in instructional responding — improving the success of instructional 
decisions. Not alt pupils improved, however. 

Strategies that improved compliance in instructional programs were less 
successful, since it was often difficult ta distinguish noncompliance from response 
deficits. Strategies that were successful included changing consequences for correct 
(compliant) responses, providing a variety of consequences for correct behavior, and 
reducing reinforcement ta variable ratio schedules. Surprisingly, however, the most 
successful strategy for improving compliance in instructional programs was moving the 
pupil ta more difficult material or higher skill levels. Some teachers eventually moved 
the pupil through six curricular steps before finding the appropriate instructional level. 
It was hypothesized that instruction at a relatively eas>* skill level might either be the 
result of noncompliance during nitial assessment or the result of the loss of interest 
by the pupil. Pupils who were required ta perform easy tasks repeatedly might become 
bored and thus fait ta "comply" regardless of the promised consequence for correct 
responding. Satiation in combination with boredom may also produce noncompliance, 
or, manipulation of teacher behavior may serve ta reinforce the pupil for noncompliant 
responding. A grant designed ta futher investigate methods of identifying ond 
remediating compliance problems has now been funded and should provide some 
information on those questions. 

Despite the success of the decision-rules, by the end of the fourth project year, 
it was clear that there was a major problem in their application. The rules suggested 
that, ff the trend for the pupil's correct performance for the five or seven day 
"decision period" were accelerating, then no change be made in the instructional 
strategies. New lirves-of -progress were drawn every five ta seven data days. 
According ta this method, however, a pupil could show accelerating trends over 
several five day periods, but his ending performance at the concluding period could be 
lower than his ending performance over the first period— (i.e., trends would appear as 



ERLC 



8 




). tn addition, this method required the teacher to make a decision 



every five ta seven days, fallowing a!! af the rule-procedures— a time-consuming 
method if no change were required after all. 

In order to remedy that situation, the decision-rules were integrated with the 
minimum 'c deration procedure (Liberty, 1972; White & Harirtg, 1976). This procedure 
allows the teocher ta specify the minimum rate af change required ta insure that ?he 
pupil's current level af performance improves ta criterion levels. A line is drawn on 
the chert af instructional performance ta indicate the desired rate af chonge. 
Guidelines for the application af these rules far rate, duration, and latency data are 
attached ta this report, tn brief, the procedures for application af the revised rules 
are: 

1 . Apply minimum 'eeleratien procedure 

a. Collect and chert three days af data. 

b. Draw a minimum 'celeration line from midpoint af the three days ta 
the intersection af the performance aim and aim date. 

c. Continue collection af data. 

2. Determine if chonge is needed 

a. If pupil meeting minimum 'celeration, na change is needed (in mast 
cases). 

b. If pupfl fails ta meet minimum 'celeration far three consecutive data 
days, change is required. 

3. Determine type af performance problem 

a. Draw Hne-of-progress for the five ta seven most recent data days. 

b. Check performance data via flow-chart questions (Figures 2 f 3, & 4). 

c. Select remediation strategy (Table t). 

Activities af the fifth project year were designed ta determine if teochers ;n a 
wide variety of settings, with varying amounts af training and assistance could 
implement decision-rules effectively. Eighty-one teochers and therapists from around 
the country were trained via one af four typical training models^ textbook only, large 
group, small group and individualized instruction. Fallowing training, the subjects 
were asked ta adopt the decision-rules for use with their pupils. The teachers who did 
odopt the rules, and who provided us with pupil performance data were 68% successful 
in their decisions and also reduced the time they spent planning each week. The data 
also indicate that teachers who fallowed the suggested strategies were more 
successful than those who did not. The cost af implementing decision-rules were as 
little as 7£ per program per week. 

The following report describes the subjects involved in the replication studies, 
the training models used, the effectiveness af the training and the impact of rules 
usage on pupil performance. Detailed descriptions af each training site follow the 
main report. 



The project was able to successful!/ meet all major, objectives, except tor the 
Investigations of generalization and adaptation. However, the importance af the 
identification cf compliance as a problem affecting perhaps 30% of the population af 
the severe!/ handicapped and seriousl/ hampering educational progress cannot be 
underestimated, and the data from the final three years af the project clearly show 
that public school teachers af the severe!/ handicapped can improve their instructional 
decision-making by applying specific, low-cost decision-rules ta the data tfcey collect 
on the progress af their pupils In instructional programs 
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Table I 

DESIGNING AND CHANGING INSTRUCTIONAL FORMATS 
General Considerations 

1. Are you providing the opportunity for independent responding? Have you 
established an allowable latency period? 

2. Do you have appropriate consequences? Do you have different consequences for 
correct and incorrect responses? 

3. Are you using appropriate signals ta get the behavior started? 

4. Are your materials natural and age-appropriate? 

5. Is the setting for instruction appropriate? 

Format Considerations for Acquisition 

1. Reinforce accurate performance. 

2. Provide sufficient response opportunities. 

3. Provide as little assistonce as possible. 

4. Provide os little extra information as required. 

5. Consider the entire Jxivtor. 

6. Consider general tzat ion when you choose constant or varied stimulus events. 

Strategies for Acqusition Problems 

Provide additional information as a 
consequence for error responses 
antecedent to the opportunity ta respond 
or conjugate with responding 

1. Ctvinge verbal /signed/gestural direction or signal. 

2* Add gestural cues. 

3. Add verba! cues/stress key words. 

4. Add a permanent model. 

5. Add a manager demonstration. 

6. Add physical prompts. 

7. Add an assisted demonstration. 

8. Add positton/calor /emphasis cues ta instructional materials. 
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Format Considerations for Fluency-Bui iding 

1. Reinforce fluent performance. 

2. Increase opportunities to perform the behavior (drill & practice). 

3. Increase the freedom to perform the behavior. 

4. Increase the rote of teacher-presentation. 

5. Consider generalization. 

Strategies for Fluency-Building Problems 

1. Provide directions for fluency before/during performance. 

2. Add/change consequences to motivate correct /fluent responses. Use o 
•Yeinfarcer survey" to identify possible consequences; provide "variable" 
consequences, o different type each time; hide the prize until it is earned; use 
conjugate consequation. 

3. Change the schedule for consequation of correct/fluent responses. Increase the 
ratio or interval of the schedule; use o variable schedule; deloy conjugate 
consequation. 

4. implement o changing aim strategy, increasing the omount of behavior required 
to receive consequction daily; tell or show the student how much "work" she/he 
must finish. 

5. Increase the number of response opportunities* 

6. Increase practice/drill. 

7. Work with o more competent peer. 

8. Add/change consequences for d is fluent /incorrect responses. 

Strotegies for Compliance Problems 

I. Move to o more difficult skill level. Are you sure this skill level is the correct 
one? Are your criteria too high? Do you require o lot of days ot criterio? is the 
student bored? 

2* Change or add o motivating consequence for correct/fluent performance. 

3. Change the schedule for consequation of correct responses to avoid satiation. 

4. Institute o response cost procedure (Cain for cor *ct/fluent» lose for 
error /disfluenl). 

5. Eliminate completing consequences. 

6. Add o time limit for no responses/change or add o motivating consequence for no 
responses and for errors (Caution: parental /guard ion permission advised prior to 
implementation of aversive, negatively reinforcing or punishing consequences). 

7. Institute "oil day" procedures for compliance. 

8. Avoid: moving to an easier skill level; repeating the instructions; coaxing or 
prompting the response; providing assistance; completing the response yourself; 
threatening the student; warning the student. 
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Fifth Project Year 
Introduction 

Activities for the fifth project /ear were designed to provide information on two 
main questions: I) Would the decision-rules prove useful to teachers and therapists in 
public school situations? 2) What type of training that would best communicate the 
decision-rules? During the first years of the project, cooperating teachers and 
therapists worked closely with members of the research staff and received extensive 
training on the application of the procedures to their pupils and instructional 
situations* Since such training would not be available outside of the research project, 
fifth year activities involved training a number of teachers in the use of the decision* 
rules using typical training models only— workshops, large groups, small groups, 
individualized instruction, and the simple availability of textbooks* 

The effectiveness of the training was measured by the subjects' ratings of the 
training and of their attitude toward the decision-rules following training* Training 
effectiveness was also measured, however, by whether or not the subjects actually 
adopted the decision-rules for use in their classrooms, it was hypothesized that in 
addition to the effectiveness of the training, adoption of the decision-rules would 
depend on the nature of instruction and evaluation practiced by the subject; if current 
practices needed to be extensively modified in order to apply the decision-rules, we 
expected it would be less likely that the teacher would adopt the procedures. 
However, if teachers did adopt the decision-rules, it would suggest that the procedures 
could be used by teachers outside of the more controlled research applications of the 
last five years. 

Data on the progress of pupils in instructional programs conducted by teachers 
who adopted the decision-rules would provide information on the impoct of the 
decision-rules on pupil progress and on the effectiveness of teacher decisions. Those 
data would help determine if the results from the first project years were unduly 
influenced by the frequent contact with research staff members or were limited to the 
populations of teachers and pupils who cooperated in previous project years. 

Most research data during the fifth year were gathered using questionnaires. 
The use of questionnaires imposed a dependence on the goodwill of subjects to return 
the questionnaires through the mail and meant that at least some questions would only 
be answered with indirect data (subjects' ratings) rather than on the basis of direct 
performance data. Many teachers applying the decision-rules did, however, share 
actual pupil progress data, thereby strengthening the basis for most of the conclusions 
reached. 

Site Selection 

There were several factors involved in the selection of sites for replication 
activities. Since much of the success of the effort would depend on cooperation^ sites 
with history of cooperation with the University of Washington were considered first. 
Secondly, the feasibility of providing training and follow-up to the sites considered. 
Since it was felt that administrative support for the utilization of the experimental 
procedures was important, we also considered the likelihood of that support for 
materials, release time for training and for social support or praise. We were also 
interested in working both with teachers who had previous training in Precision 
Teaching and at least some who had not. 

erJc is 
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After some deliberation, initial contact was mode- with the following persons: 
Dr. Bill Tilley (Seattle Public Schools), Dr. Stillman Wood (Olympta Public Schools), 
Dr. Wayne Sailor (San Francisco State University), Mr. Carl Binder (Fernald School, 
Boston) and Ms. Vol Lynch (Center for Inservice Training and Program Development, 
University of Washington). Dr. Sailor recommended a school for which he and his staff 
had provided training. When contocted, the administrator of this school was willing to 
participate, but later did not respond ta letters or calls. Arrangements to work with 
each of the other sites were successful (see individual Site Reports for details). 

In Seattle, Dr. Tittey ogreed to continue the cooperative arrangement initiated 
during the fourth project year, and suggested that project staff work individually with 
building principals in Identifying potential subjects. Since Seattle Schools, and the 
Special Education Department in particular, have a long history of involvement with 
the University of Washington, and since Precision Teaching advocates were formerly 
found in the Washington State Office of Public Instruction, it was expected that a 
majority of consenting subjects would have had previous training in Precision Teaching 
techniques. 

Dr. Sttllman Wood, Director of Special Services for the Olympia School District, 
has long advocated and supported the use of Precision Teaching procedures in that 
district. He was eager to participate, especially since his teaching staff were 
requesting training in decision-making procedures. Dr. Wood introduced us to the two 
advisors who provided direct supervision, support and data review for teachers of the 
severely handicapped in the district. Dr. Wood, and the advisors (plus the advisors for 
the other special education teachers in the district) were trained in the application of 
the experimental procedures prior to the initial meeting with potential teacher 
subjects. In fact, 6 of the 7 consenting teachers, and 2 of tKe k therapists reported 
previous training in Precision Teaching. 

Mr. Carl Binder, of the Behavior Prosthesis Department of Fernald School in 
Boston, had been trained in the use of Precision Teaching by Dr. Ogden Lindsley. As a 
result of his interactions with various teachers throughout the New England area, both 
in preservice and inservice consultations and Gaining, Mr. Binder hod developed an 
informal "sharing" group of Precision Teaciiers in the New England area. Mr. Binder 
writes a monthly open letter, "Data Sharing Newsletter", and conducts monthly dota- 
sharing sessions for teachers in the immediate Boston area. He also keeps in touch 
with teochers via telephone ond moil contacts. Mr. Binder agreed to act as an 
intermediary between the IH project and those teachers. He sent a list of potenttc! 
subjects, which the IH project then contacted. The IH project sent Mr. Binder 
Handbooks for consenting subjects. Mr. Binder distributed the Handbooks* and ogreed 
to prov*de any follow-up assistance requested. 

Ms. Valerie Lynch is the project coordinator of both the Center for the Severely 
Handicapped and the Western Region Inservice Training Center, two grants funded by 
the U.S. Office of Special Education. With the support of the Washington State Office 
of the Superintendent of Public Instruction, and the Area of Special Education, College 
of Education of the University of Washington, the staff of those two projects, under 
the auspices of the Center for Inservice Training and Program Development (CITPD), 
.provided two separate training workshops for personnel working with the severely and 
profoundly handicapped. In generai, school districts were invited to send an 
"Interdisciplinary Team" (teacher, administrator, a member of a support profession, 
therapist, nurse, and another teacher or member of a support profession) to one of the 
two workshops. Eoch team contracted with the CITPD to attend th2 Summer 
Workshop, and to develop program goals and objectives based on a needs assessment 
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carried out by CITPD staff. The CITPD then provided the inilial training aid follow- 
up designed to accomplish the Individual objectives for each project site* At the three 
week workshops held in Seattle* participants generally spent half of each day attending 
didactic sessions on a variety of topics (including a modified Precision Teaching 
approach) and another part of the day in supervised practlca with severely and/or 
profoundly handicapped pupils. With the cooperation of Ms. Lynch, participants in 
those two workshops were invited to become subjects in the Instructional Hierarchies 
Project* 

The initial differences among th* sites in the areas of estimated direct 
administrative support for the use cf procedures* the number of subjects with a prior 
training in Precision Teaching, and ttv* Trture of the training and follow-up provided to 
each site are shown in Table 2< Together f consenting subjects at the sites were 
employed in a total of 18 public schools (of which 12 schools serve both handicapped 
and nonhandicapped pupils) in 13 different school districts and 7 private day or 
residential schools and 3 State Residential Schools. 



Table 2 

INITIAL DiFFRREK :ES BETWEEN SITES 

Site Estimation af 

Administrative 
Support for use af 
Procedures 



Percent af consenting Nature of Training Nature of 

Subjects Reporting Plonoed Follow-up Planned 

Previous Training in 
Precision Teoching 



Site I Varied from school to 
school; high support 
available from Project 
Intermediary 



77% 



Textbook Model : 
Subjects previously trained 
in Precision Teaching by 
Project Intermediary. 
Project provides textbook 
for use of decision-rules. 



Follow-up available 
from Project 
Intermediary and 
research staff 



Site 2 



Site 3 



Site k 
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Administrators supported 
participation in CITPO 
Workshop. 



16% 



Administrator required use af 
Precisian Teaching techniques 
and provided taservic<- training 
and ongoing support. 



General odministrotive support 
for procedures, little or no 
direct support within schools. 



73% 



66% 



Large Group Model : 
Intensive training in edu- 
cational technology related 
to education of sph pupils, 
including use of modified 
Precision Teaching procedure 
provided to all participants 
by CITPD. Three large group 
sessions on use of decision- 
rule procedures provided by 
IH Project. 

Small Group Model: 
Training In use of experi- 
.nental decision-rules 
provided in smalt group 
session. 

Individualized Train i ng Modeh 
Intensive individual 
training of Precision 
Teoching procedures and 
experimental decision-rules 
provided by IH staff. 



Follow-up provided 
by CITPD staff 
with input from 
iH Project 



In person and 
telephone contact 
upon subject request 



In person and 
telephone contact 
upon subject request. 



2? 
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METHOD 

Subjects ot each site were introduced to the project in different ways, although 
each received essentiolly the same information. Potential subjects of Site I were 
identified by Mr. Carl Binder, who provided names and addresses to the project. 
Participants in the Workshops conducted by CtTPD (Site 2) were also identified by 
perrons other than the IH staff or district personnel. The potential subjects in Sites 1 
and 2 each received tetters describing the project. In addition, the Workshop 
participants attended an oral presentation concerning the project. Potential subjects 
at Sites 3 and k were identified by the administrators of the districts involved, and 
attended a meeting with the research staff concerning the nature of the project. 
(Details of the meetings and copies of the tetters and consent foi'ms are provided in 
the individual Site Reports which follow the general report.) 

A total of 122 potential subjects were contacted. Of these, 107 (88%) returned 
consent forms, signed or unsigned. Of those returned, 89 (83%) consented to 
participate. The general method employed by the project is outlined in the nine steps 
below. Differences among sites, and the exact nature of the questionnaires and other 
methods for data collection used, are described in succeedirtg sections of this report. 

I. Consenting subjects provided background information concerning relevant 
teaching experineces and personnel demographics. 

2* Consenting subjects from all sites, excepting Site I, trained in the use of the IH 
procedures, and completed a questionnaire concerning their evaluation of the 
training. 

3. Additional information requested by the subjects concerning the content of 
training was provided, at all sites except Site I. 

4. Each subject receive * the Handbook of Experimental Procedures . 

5. Each subject received a questionnaire requesting their evaluation of the 
Handbook . 

6. Each subject was asked whether or not they intended to use the experimental 
procedures with their pupils. Subjects who dio not intend to use the procedures 
were asked for their reasons. This concluded their participation in the study. 

7. Follow-up assistance for the implementation of the IH procedures and any other 
related material was made available to those subjects who decided to try the 
experimental procedures. Availability of that assistance was continued 
throughout the project. 

8. After 4-6 weeks, subjects remaining in the study were asked to evaluate the 
effectiveness of the procedures, and to provide additional information. With 
subjects deriding to continue in the study, identical information was requested 
on a periodic basis in order to determine attitude change, if ony. In addition, 
actual pupil performance data were requested of the subjects. 

Subjects were free to withdraw from the project at any time, without penalty of 
any kind, ond were free to leave any question or item unanswered on forms provided by 
the project. Every effort was made to keep the identity of participating subjects 
unavailable to their supervisors, in accordance with the provisions of the University of 
Washington's Rights of Human Subjects Committee. „ 

2j 
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In generol, data were collected through the use of questionnaires, although pupil 
per form otk *- data were made available to the project by several subjects. Severe 
timitotions are, of course, placed on research conducted through the use of data from 
questionnaires, and will be discussed in the "Data Collection* 1 section of this report, as 
well as in the Results and Discussion sections. 



Subjects 

The subjects of ine project were 69 teachers and 1 6 therapists' serving a total of 
t, 215 handicapped pupils. Although the experimental procedures were designed for use 
by teachers serving primarily severely and profoundly handicapped students, we found 
that classrooms usually included a mixture of handicapping conditions) mast notdbly, 
moderately handicwped pupils were integrated with severely and/or profoundly 
handicapped pupils. (The reported handicapping conditions of pupils served by the 
teachers and therapists are shown in Table 3). POpiJs identified by their 
teachers, therapists as severely, profoundly or multiply handicapped constituted 
approximately 50% of the siudents (N=60l), with the addition of the moderately 
handicapped pupils this total rises to about 74% (N?904) of the pupils reached by the 
consenting subjects. Each of pupils had the potential of providing information on the 
impact of the experimental procedures on pupil progress In instructional programs. 

The ages of the Individual pupils are not known, but the ages served by each 
teacher /therapist are shown In Tables 4 and 5. It is interesting to note that Sites I and 
2, which served a much h;gher proportion of institutionalized students, included more 
classes serving mixed ages ranges (e.g., 6-22; 1 1-54) and, of course, odults, than did 
the two sites which were exclusively public school settings, Site 3 and 4. The oge 
range served by the subjects was from I through 54. 



Although not originally planned, three administrators asked to participate. The 
odministrotors were interested in teaching their personnel to use the procedures. The 
administrators from Site 2 decided not to da so following training, and the Site I 
administrator^ teachers consented to serve as subjects themselves. Data from the 
odministrotors is not Included in this section. 

2 

Information concerning the number and type of pupils served was provided by the 
subjects on the Background Questionnaire. Research staff personnel did not attempt 
to verify that information in any way. 
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!e 3 

HANDICAPPING CONDITONS OF PUPILS SERVED BY SUBJECT TEACHERS AND THERAPISTS 

Subjects Learning Mildly Moderately Severely Profoundly Multiply Other Total 1 

Teochers Disabled Handicapped Handicapped Handicapped Handicapped Handicapped Pup) Is 



Site 1 


N=27 


L 


9 


36 


93 


56 


9 


38 Z 


243 


Site 2 


Nz23 


10 


25 


33 


85 


42 


7 


2 3 


204 


Site 3 


N=7 


0 


7 


23 


18 


14 


0 


0 


62 


Site 4 


N=I2 


8 


14 


68 


76 


32 


II 




215 



Total N=69 20 55 140 272 144 27 46 724 
Therapists 

Site 2 Nz12 29 4? 118 60 46 4 76 5 382 

Site 3 N=4 3 17 25 44 10 0 10 6 t09 

Total N=I6 32 66 143 104 56 4 86 491 

TOTAL N:85 52 121 303 376 200 31 132 1215 

'calculated from the nurrbers provided per category* usually corresponding to jmber served per year, but not always. 

*8 cteaf /hearing-impaired and 30 autistic 
3 

or t hoped! calty handicapped 
**6 ort hoped leal I y handicapped 

^?0 communication disorders, 6 orthopedically handicapped 

*I0 communication disorders 26 
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Table 4 

AGE GROUPS OF PUPILS SERVED BY TEACHERS 



Pupil Age Grew* 



Site I 



Site 2 



te3 



Site 4 



TOTAL 



Preschool: 2-6 
Classrooms 
Pupils' 

Elementary 6-12 
Classrooms 
Pupils 



2 
19 



5 

35 



5 
6\ 



2 

27 



3 

23 



2 
41 



5 

39 



9 

103 



\6 
142 



Mixed 2.12 

Classrooms 
Pupils 

High school 12-22 
Classrooms 
Pupils 

Mixed 6-22 
Classrooms 

Pupils 

Adults 22+ 
Classrooms 
Pupils 

Mixed 1 1 + 

Classrooms 
Pupils 



2 

2k 



6 

68 



I 

5 



I 

4 



15 



7 
16 



3 

30 



2 
18 



2 
13 



I 

5 



4 

127 



3 

29 



19 
284 



4 
35 



3 

22 



15 
134 



Calculated from the number reported served per year. 
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Table 5 

AGE GROUPS OF PUPILS SERVED BY THERAPISTS 

Mixed 1-2 /ears 

Therapists - 2 

POpils 1 - 75 

Mixed I -1 8 year > 

Therapists - 2 

POpils - W 

Mixed 3-22 /ears 

Therapists 12 
POpils 405 

'Calculated from the number reported served per /ear. 

Class Size was computed from the number of pupils the teachers reported 
serving an an average da/. The 69 teachers reported serving an average of 680 pupils 
per day* as compared with 750 pupils over the course of the school year. Differences 
in those totals may be attributed to obsences, deaths, and placement or school 
transfers. On an average day, the mean number of pupils served by a teacher was 9.9 
(range 2-50). Five of the 69 teachers each served more than 20 pupils per day. Two 
teachers are resource room teachers serving 20 and 50 mildly handicapped pupils, one 
teacher served 28 moderately handicapped pupils (ages 13-17) with one part time 
assistant, one teacher was involved in a team teaching situation with two other (non- 
subject) teachers serving 36 preschool children, and one teacher conducted a 
vocational training program based on a resource room model for 50 sph students. 
Excluding those five teachers- the remaining 6k teachers served 569 pupils per year, 
and 51 1 on an average school Hay, with a mean class size of eight (range 2-16). Class 
sizes for the different sites are shown in Table 6. The number of pupils receiving 
direct service from the 16 therapists totaled 529 per year, while the number per day 
was about 1 88 total (mean 1 1 .8, range 5*20) (see Table 6). 

Table 6 
CLASS SI2E 





Teachers 


Pup t Is/ Year 


Pupils/Day 


Mean/Day 


Range 


Site 1 


26 


226 


198 


7.6 


2-12 


Site 2 


22 


200 


178 


8.1 


5-14 


Site 3 


7 


63 


59 


8.4 


6-12 


Site 4 


9 


80 


76 


8.4 


5-16 


TOTAL 


64 


569 


St I 


8.0 


2-t6 



' Excluding teachers who team teach or who serve pupils in a resource room model. 
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Site 2 
Site 3 



Table 7 
CLIENT LOAD 

Therapists Pupils/ Year Pupils/Day Mean 'Day 

12 405 148 12.3 

4 124 40 7,6,10,17 



Range 
5-20 



TOTAL 



16 



529 



188 



11.8 



5-20 



Previous Experience . All but twa of the teachers had previous experience 
working with handicapped pupils, with a mean number af 4.7 years (range 1-22), and all 
af the therapists alsa had experience working with the handicapped, with a mean af 6.3 
years (range 1-23). Experience af the teachers and therapists ore shown for each site 
in Table 8. 



Table 8 

PREVIOUS EXPERIENCE OF SUBJECTS 



Subjects 

Teachers 

Site I N^27 

Site 2 N=23 

Site 3 N-7 

Site 4 N=I2 



N with previous 

teaching 

experience 



27(100%) 
22(96%) 
6(66%) 
12(100%) 



Years 
experience 
mean (range) 



3.8(1-22) 
5.8(1-28) 
6.7(1-13) 
9.6(3-22) 



N with previous 
special ed. 
experience 



27(100%) 
22(96%) 
6(86%) 
12(100%) 



Years 
experience 
mean (range) 



2.9(1-7) 
5 (1-21) 
4.4(1-10) 
a3(3-22) 



Total N=69 
Teachers 



67(97%) 



5.7(1-28) 



67(97%) 



4.7(1-22) 



Therapists 

Site 2 N:I2 

Site 3 N=4 



12(100%) 
4(100%) 



7.2(1-23) 
4.3(1-8) 



12(100%) 
4(100%) 



6.9(1-23) 
4.5(1-8) 
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Total N=I6 
Therapists 



16(100%) 



6.4(1-23) 



96(100%) 



2P 



6.31(1-23) 
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Assistance Available . Sixty-three of the 67 experienced teochers hod some help 
in the classroom from one or more of the following: paid assistants, volunteers, 
student teochers or other personr el (e.g., parents). There were o totol of 127 
assistants, 38 volunteers, 55 studeni teochers ond 77 other types of assistants; the 
mean number of people helping one teacher each week was 4.2 <ra.ge 1-22). On the 
average, teochers who received assistance had the equivalent of 47.3 hours of help per 
week (range 4-122 hours). The fou< teachers who did not have any assistance in the 
classroom served a total of 16 severely handicapped pupils, 8 moderately handicapped 
pupils and four profoundly handicapped pupils, with oges ranging from 12-54* (See 
indiviudat site reports for more details.) 

Nine of the 16 therapists had some assistance. A total of 16 helpers served those 
nine thearpists for an average of 39.3 hours per week (range 5*120). Four of the 12 
therapists from Site 2 were without help, and three therapists from Site 4 did not have 
any assistance available. 

Therapy Assistance Available to Classroom Popils . Fifty-ei^ht of the 69 
teachers reported that therapists worked with at least some of their pupils eoch week, 
for on average of eight hours per week (range 1-40). The nine teachers who reported 
that no therapists were working with their students served a total of 110 pupils per 
year (15% of the pupils served by the teachers). Six of these nine teachers were from 
Site t, and ei^ht of the nine teochers served pupils older than 12 years. 

Instructional Practices 

Types of Instruction . All of the teachers conducted instructional programs 
designed to increase behavior, either by teaching new behavior and skills or by 
improving performance in skills ond behaviors which the student hod previously 
acquired ond which included some opportunity for individual pupil responding. The 
majority of instructional time was spent in 1:1 situations, although only three teochers 
provided 90% or mere of their instruction in that situation. The least amount of 
instructional time, as can be expected, was spent in group situations requiring unison 
responding, although individual responding in group settings was also used by most 
teachers. Aj a group, therapists conducted most of their programs in hi situations, 
with three of the 16 therapists using this type 90% or more of the time (Table 9). 
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Table 9 

TYPES OF INSTRUCTION PROVIDED BY TEACHERS 

individual Response Opportunities Site I Site 2 Site 3 

L During 1:1 instruction 

% teachers who provide: 96% 100% 100% 

Mean amount of instruction time: 53% 41% 55% 

2* At natural occasions 

% teochers who provide: 77 91 100 

Mean amount of instruction time: 41 26 21 

3* In group settings 

% teachers who provide; 81 86 86 

Mean amount of instruction time: 22 27 21 

Unison Response Opportunities 

% teochers who provide: 26 55 29 

Mean amount of instruction time: 8 16 15 

Table 10 

TYPES OF INSTRUCITON PROVIDED BY THERAPISTS 



Individual Response Opportunities 

I* During 1:1 instruction 

% therapists who provide: 

Mean amount of instructional tierm 

2. At natural occasions 

% therapists whc provide: 

Mean amount of instructional time: 

3. In group settings 

% therapists who provide: 

Mean amount of instructional time: 

Unison Response Opportunities 

% therapists who provide: 

Mean amount of instructional time: 



Site 2 



100 
71 



67 
II 



92 
20 



42 
6 



Site k 



100% 
36% 



92 
31 



100 
31 



25 
7 



Site 3 



100 

95 



25 
5 



25 
10 



Total 



97% 
06% 



88 
35 



85 
25 



36 
13 



Total 



100 
77 



56 
10 



75 
19 



0 
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Frequency of Instruction . The frequency of instruction conducted by teachers is 
shown In table ID. Generally, most instructional programs were conducted at least 
once per day, although one subject reported conducting programs twice per week and 
eigjht conducted instructional programs three times per week. Therapists, on the other 
hand, usually conducted instructional programs twice a week (Table 1 I). 

PuPil Response Opportunities . Most (67) of the subjects provided 15 or fewer 
response opportunities when they conducted instruction! 26% provided 1*5 trials, 17% 
provided 5-9 trials, and 32% provided 10-15 trials (Table 12). 
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FREQUENCY 
Per Day 

Natural occasions 
throughout the day 

Two or more preset 
sessions per day 

One preset 
session per day 

Per Week 

Four preset 
sessions per week 

Three preset 
sessions per week 

Two preset sessions 
per week 

One preset session 
per week 



Site t 



17 



Site 2 



13 



3 
I 



Table 1 1 

FREQUENCY OF INSTRUCTION 
TEACHERS 
Site 3 Site k Total 



I 



8 



II 



43 



8 
I 



Site 2 
0 
I 

2 

0 
I 

S 



THERAPISTS 
Site 3 



TOTAL 



I 

2 
I 



Total 



0 2 

1 , 12 

4S 



0 
2 
7 
S 



3 
10 
8 
S 



One therapist "sees" pupils either one, two or three sessions per week. 
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Table 12 

PUPIL RESPONSE OPPORTUNITIES PROVIDED DURING INSTRUCTION 
Number of Response Opportunities Site 1 Site 2 Site 3 



Site 4 



Per Instructional Session 

1-5 

5-9 
10-15 
16-20 

20+ 



8 
0 
12 
2 
2 



Teachers Therapists Teachers Therapists 

6 3 1 0 3 

5 3 1 2 3 

10 3 2 0 4 
0 2 1 0 2 

011 II 



Total 



21 
14 
31 
7 

6 
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Evoluotion of Popil Performonce 

Number of Programs . Thirty-seven of the subjects collected data on 9S-I00% of 
their programs, and 18 collected data on 85-94% of their program (Table 13). Seven 
subjects collected data on less than 25% of their programs. Generally, teachers at 
Sites I and 4 seemed more likely to collect data an almost all programs than teachers 
at other sites. 

Table 1 3 

PERCENTAGE OF PROGRAMS ON WHICH PUPIL PERFORMANCE DATA COLLECTED 



Percentage of Programs Site I Site 2 Site 3 Site 4 Total 



95-100% 


21 


9 


6 


3 


39 


85-9*% 


2 


II 


2 


3 


19 


75-84% 


1 


3 


1 


1 


6 


50-7*% 


2 


4 


2 




.8 


25-49% 




2 




2 


4 


10-24% 


1 


1 




2 


4 


less than 10% 




3 






3 



Frequency . Teachers collected data mare often than therapists. Thirty-six 
teachers and three therapists collected data every time an instructional program was 
conducted. Twenty-seven teachers and four therapists collected data almost every 
time a program was run, and four teachers and four therapists collected data about 
half the time the program was run. Four therapists collected data for initial 
assessment purposes only. 

Number of Trials . Thirty-two teachers end five therapists collected data on 
every trial when they collected any data at all. Ten teachsrs and one therapist 
collected data an ail trials during a specified period of time. Other subjects collected 
data on only specific number of responses (Table 14). 
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Table 14 

NUMBER OF PUPIL RESPONSES EVALUATED 

Frequency Teachers Therapists 

data collected on eve// trial 32 5 

data collected on first trial only 3 I 

data collected on last trial only 5 

data collected on random number 6 4 

data collected on probe trials only 6 4 

data collected on all trials during 
specified period af time 10 ) 

mixed 7 

Type af Data . Teachers generally preferred either accuracy data or rate data 
(Table 15^ clearly ranking those two data types higher than the others. Therapists 
showed less af a clear cut preference* although six ranked behavior counts as the data 
type they used most frequently. Teachers af Site I and Site 3 showed a strong 
preference for rate data* while teachers from Sites 2 and 4 preferred accuracy data. 

Use af Data . Forty teachers (60%) and four therapists (25%) graphed data (all 
teachers af Site 3, 22 af 27 af Site I* 3 af 1 2 af Site 4 and 8 af 23 af Site 2). The four 
therapists who graphed data were from Site 3. All but two subjects hod rules for use 
in making instructional decisions in at least some of their programs. Seventy-six had 
rule* or criteria for determining when the pupil met the performance aim, and 81 had 
rules for deciding what ta do after the p^pil met the aim. Sixty used specific criteria 
in deciding when ta move the pupil bock ta an easier skill level, Thirty-nine reported 
rules for deciding when ta change instructional procedures. Surprisingly, 37 reported 
rules for determining what type af intervention ta make when changing instructional 
procedures. 

Planning . The 67 experienced teachers spent an average af 4.66 hours per week 
planning instructional programs (range 1-20 hours), while therapists spent an average 
af 2.63 hours per week (range 1*8 hours). The 37 teachers with training in precision 
teaching spent, on the average, 45 minutes more per week planning than the 30 
teachers without such training. 
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Doto type 



Accurocy data 
Rote date 
Count of behavior 
Count of trials 
Levels of assistance 
Other time-based data 



N Use 
U6 
k9 
37 
27 
38 
38 



Table 15 

TYPES OF DATA COLLECTED BY SUBJECTS 
Teachers 



N Ranked 1st 
28 
2k 

10 

5 

3 

0 



Mean Rank 
1.87 
1.86 
2.18 
2.7 
2.9 
3.47 



N Use 
10 
9 
It 
9 
9 
9 



TVierqpists 



N Ranked 1st 
k 
I 

6 
I 

2 
I 



Mean Rank 
2.6 

2.55 
2.18 
3.U 
2.55 
3.88 
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Training and Follow-up 

All subjects were provided information about the application of the experimental 
decision-rules. Subjects from Site I received only the Handbook of Experimental 
Procedures . Subjects from other sites were trained by project staff. Subjects from 
Site 2 were trained in a large group, Subjects from Site 3 were trained in a small 
group, and subjects from Site 4 received individualized training. The content of the 
training varied somewhat from site to site, based on some changes in the experimental 
procedures and according to subject request. 

Content of Training 

The decision-rules which were derived during the first four years of the project 
provided the bas * for all training. All subje^s were provided with descriptions of the 
prerequisites for using the decision-rules: 

1) Provision of instruction designed iu teach the pupil c -*ew skill, or to improve the 
fluency of performance of a previously learned skill. Se decision -rules are not 
designed for programs designed to decelerate behavior* 

2) Selection of a criterion for performance of the behavior. In some coses, it would 
be necessary for subjects to select bath a terminal criterion performance level 
and a target date for reaching that criterion. 

3) Opportunity for the pupil to respond individually to the instructional cues. 
Unison, or group responding, would not provide the data necessary for application 
of the rules. Further, it was recommended that the pupil have at least ten 
response opportunities per school day. 

k) Collection of performance data. Subjects were encouraged to collect either 
rate, latency or duration data, although subjects from Site 2 were also taught by 
the CITPO staff during the workshops to use the decison-rules with percent 
correct data. 

S) Graphing performance data for analysis on semi logarithmic charts, either the 
standard Behavior Chart used in "Precision Teaching" or a modification of that 
chart used for percent correct data. 

Following training in the prerequisites, which varied considerably frtm site to 
site, subjects were provided information on the use of the rules. The procedures 
presented to Site 2 subjects were different from those provided to subjects at other 
sites. Site 2 subjects were trained prior to the conclusion of fourth year activities 
(June, 1979). Thus, they were provided rules based on "performance patterns", which 
entailed drawing Hnes-of-progress every six data days, and then using a flow-chart to 
select a strategy change. After the analysis of fourth year data, however, it was 
decided to integrate the "minimum 'celeration rule" (Liberty, 1972; White & Haring, 
1976) with the experimental decision-rules. Thus, subjects of Sites 2, 3 and k received 
information on how to draw minimum Vreleration lines; drawing Hnes-of-progress only 
when the pupil fell three days below that line, and then using a flow-chart to select a 
strategy change (Figure 2-4). Subjects from Site I were later provided with the rules 
for use with minimum 'celeration. After training in the use of the decision-rules, 
subjects were provided information on the types of strategy changes recommended for 
each of the performance categories. Decision-rules for use with programs designed to 
accelerate duration (Figure 3) or decelerate duration or decelerate latency (Figure k) 
of responding were mailed to all subjects upon completion of those rules. 

The Handbook of Experimental Procedures consisted inittolty of the following 
information: 
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(1) Step-by-step directions for applying the decision-rules. 

(2) The decision-rules for use with programs designed to accelerate rate of 
responding, and to accelerate percentage correct (Site 2 only). 

(3) Lists of the strategies recommended for use. 

ik) Descriptions required for application of the rules, including: drawing iines-of- 
progress, drawing minimum 'celeration lines, determining percent correct, 
determining H$\ variability and determining sharp deceleration in corrects. 

(5) Text descriptions of instructional formats and instructional strategies 
recommended for use as intervention. 

Additional information added to the H andboo k during the lost year included: 

(6) A description of the prerequisites for use of the decision-rules. 

(7) A description af methods of collecting and charting rate, adjusted rate, duration 
and latency data. 

(8) Rules for use with latency aid duration programs. 

Subjects who had completed training, and were still participating in the project, 
received the information in points 6-8 as part of the Handbook . 

Type of Training 

Subjects at sites 2 through 4 were trained according to different models. Site 2 
subjects were trained in the use of the experimental decision-rules as part of a 
summer workshop conducted by CITPO. Participants attended didactic sessions and 
worked with pupils in supervised practica. The workshops were conducted for a total 
of \k days over a 3 week period, usually lasting from 8 ajn. - 5 p.m. daily. 
Participants received university credit upon satisfactory completion of the workshops. 
Alt participants of the workshop attended a one-hour session in the application of the 
decision-rules with percent correct data, which included directions for applying the 
rules and drawing !ines-of-progress for sample data. Subjects of the !H project 
attended two additional sessions, offered a total of four times, which included more 
detailed information on the application of the rules and the instructional strategies. 
Those sessions usually included opportunities for practice of the procedures (see the 
Site 2 Report for a complete description). 

Site 3 subjects were trained in the use of the decision-rules in a s mall group 
training model t although subjects who missed one of the two sessions were trained in 
individual or smaM group "make-up" sessions (see the Site 3 Report for a complete 
description). Subjects at Site 3 were trained in two one-hour sessions, conducted offer 
their pupils had lef i school for the day, during the Fall of 1979. Subjects practiced 
applying the ruies for one, two or three practice examples only. 

Site k subjects were trained in the use of the decision-rules in a individualized 
training model . These subjects met individually with a member of the research staff, 
who directed training toward specific example of programs within the subjects? 
classroom. Training sessions were scheduled at the subjects 1 convenience. Four or 
five thirty-minute training sessions were usually held with cC*:h subject. 

No direct training was provided for subjects at Site I. Each of those subjects 
was simply sent a copy of the Handbook . 
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Length of Training 



The length of training time varied from site to site (Table 16). The amount of 
time spent on any one topic also varied. Subjects from Site 3 spent only a few minutes 
discussing methods for collecting and charting cteta, since those subjects indicated that 
those topics were familiar to them; subjects from Site 4 spent up to \h hours on the 
same topics. The training fame for Site 2 shows the time spent in sessions conducted 
by the IH project, althwgh approximately 15 hours were also spent during the 
workshop covering related areas (e.g., conducting programs, collecting and charting 
data). 



Table 15 



LENGTH OF TRAINING 



Site Training Model Training Time in Hours/Subject 

Site I Hondbook Only (N=27) 0 

Site 2 Large Group (Nz3 1) 3.75 <N=I4) 

4.0 (N=I7) 

Site 3 SmaMGroup(N=ll) 2.0 

Site 4 Individualized (N=I2) 2.5 (1.92-3.38) 



Follo»v-up 

Information on the general application of the decision-rules and associated 
procedures was available to subjects of Sites % 3 and k following training. Those 
subjects were able to request follow-up when they completed the Presentation 
Response Questionnaire. The requested information was provided through individual 
meetings between project staff and the subject, or through telephone conversations. 
Follow -op information on the application of the procedures to specific programs was 
also available to all subjects who ogreed to odopt the decision-rules with their pupils 
for a trial period. Follow-up information was available on request provided by Mr. 
Carl Binder to Site t subjects. Information was provided either via written 
communication, personal visits, telephone conversations or some combination of 
methods. 



Data Collection 



EMC 



Data collection procedures were designed to provide information relating to the 
adoption of the decision-rule procedures by the subjects, the intact of the procedures 
on pupil performance and an estimate of their cost-effectiveness. Information 
relating to the adoption of procedures by subjects was collected through a series of 
questionnaires. Data on the impact of the procedures on pupil performance was 
provided by subjects who contributed pupil performance data and through subjects' 
evaluation of the procedures on questionnaires. Cost-effectiveness data were 
estimated by project staff for tr^ning and follow-up, and subsequently related to 

42 
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overall impact. Each af the data collection procedures are described in the fallowing 
sections. 

Q uestionnaires 

Questionnaires ore clearly limited as a method af data collection. The major 
limitations are imposed by tne type, construction, and sequencing of items on the 
Questionnaire, by the return rate, by the reliability of the data so collected, and by 
the subjective nature of any ratings provided by the subject. In order ta mirfmize 
errors caused through the construction af questionnaires, field-testing procedures were 
used for the Background Questionnaire. However, since subsequent questionnaires 
were designed ta follow training, and since field-testing populations and project time 
were limited, the other questionnaires were nat field tested prior to their use with the 
total subject sample. In order to improve the return rate, subjects were prompted by 
telephone or mail to return questionnaires. The reliability of the information provided 
by teachers was not checked by project staff. In general, a standard S point Likert 
scale was used on items requiring subject ratings. The questionnaires were 
administered in the sequence described below. Subjects were free to not answer ony 
item or quest iormatre, and free ta withdraw from the study at any time without 
penalty of any kind. 

Background Questionnaire . Items on this Questionnaire concerned information 
about the perils served by the subject, the assistance available to the subject, the 
educational background, instructional procedures and evaluation methods generally 
utilized by the subject. 

A draft of the background questionnaires was prepared during the fourth project 
year and was field tested to determine if the questions were understandable, 
answerable, and resulted the information desired. The test population consisted of 
heod teachers at the Experimental Education Unit of the University of Washington who 
volunteered to assist the project. Fifth year activities were not planned at this site, 
so that the participatior of the teachers in the field test would not compromise othet' 
research activities. A third and final draft of the questionnaire was prepared in April 
1979 incorporating revisions based on the results of twelve test subjects. A copy of 
the Background Questionnaire is included in Appendix I. 

Background Questionnaires, along with the consent form and introductory letter, 
were distributed to all potential subjects. For 3 af the 4 sites, that material was given 
directly to potential subjects by research staff members. Materials were mailed to 
Site I subjects. 

Presentation Response Questionnaire . With the Presentation Response 
Questionnaire subjects provided an evaluation of the training and were able ta request 
follow-up information cn specific topics. There were two major parts of the 
Questionnaire (Appendix Ih "clarity of information presented", and "attitude toward 
the procedures". Both sections utilized a standard Likert scale and provided space for 
written comments. Minor changes were made in the questionnaire for some sites to 
reflect the content of training provided. The PresentcWon Response Questionnaire 
was administered to all subjects (except Site I) following the conclusion of training. 

Handbook Questionnaire . The subjects' responses to the Handbook Questionnaire 
(Appendix 1) provided feedback on the clarity of information presented in the 
Handbook of Experimental Procedures . Subjects were asked to respond to questions on 
nine specific content areas of the Handbook, and to make any comments or suggestions 



33 



regarding the Hondbook in general. The questionnaire was mailed or given to subjects 
who had received a cop/ of the Handbook . Usual!/ two weeks elapsed between the 
distribution of the Handbook and the distribution of the Questionnaire. 

Trial Period Questionnaire . Following the initial consent for participation, the 
Trial Period Questionnaire (Appendix I) was the most important questionnaire as far as 
determining the willingness of the subjects to utilize the experimental procedures. 
Subjects who agreed to utilize the experimental procedures in the classroom provided 
information on the number of pupils and programs to which procedures would be 
applied and their estimation of administrative support. The/ were also given the 
opportunity to request follow-up assistance. For subjects who did not agree to 
continue their participation, several items requested the reasons for their decision. 
This questionnaire was distributed to subjects along with the Handbook Questionnaire. 

Procedures Questionnaire . This questionnaire (Appendix I) was designed to 
provide information about the impact of the experimental procedures. There were 
five parts to the Questionnaire: pupils served by the subject; usefulness of the 
Handbook; general attitude toward the procedures; impact of the procedures; and 
decisions as to further pert icipat ion and utilization of the experimental procedures. 
Changes in the subjects 1 attitudes about the procedures and the Handbook^ for 
example, may be determined through comparison with their responses to similar items 
on the Presentation Response and Handbook Questionnaires. The Procedures 
Questionnaire was odministered to subjects who agreed to continue project 
pcrticipation by utilizating the experimental procedures in their instructional 
programming. Subjects received the Procedures Questionnaire at the conclusion of the 
trial period described in the Trial Period Questionnaire or after six weeks, whichever 
came first. Subjects who selected longer Trial Periods, or who decided to continue 
project participation, then received a second Procedures Questionnaire after another 
sfx week period. Unfortunately, the conclusion of the project year prevented the 
distribution of additional Procedures Questionnaires, which could have been useful for 
monitoring continued changes if, attitude and impact over time. 

POpil Performance Data 

Subjects who odopted the decision-rule procedures were asked to provide copies 
of pupil performance data. If provided by subjects, data for each pupil and each 
program were analyzed to yield the following information: 

(1) The type of data collected by the subject. 

(2) The total number of calendar and data days included. 

(3) Whether or not the subject applied the procedures appropriately. 

(4) The number and types of auricular changes made. 

(5) The number and types of instructional strategy changes made. 

(6) The number of changes which were in accord with the rules and the number that 
were not in accord with the rules. 

(7) The effect of the strategy changes on pupil performance for the first five days 
following an intervention (immediate increase or decrease in corrects or errors, 
and chonges in direction or magnitude of trend). 

The reliabitity of the data collected on pupil performance by teachers was not checked 
by project staff. 
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Cost DotO 



Cost data were estimated for training ond for follow-up. In order to provide a 
basis for estimation, personnel costs were established according to the following 
salaries: 



(1) A twelve month salcry for a trainer was estimated to be $17,000, based on a 
salary structure provided by the University of Washington. An average 8 hour 
day for on average of 20 days per month was used to calculate the hourly rate of 
$8.85. 

(2) Based on information provided by the Seattle Public Schools, an average special 
education teachers' salary was estimated at $I8,7S4 for on 182 day contract, 
with the hourly rate calculated ot $12.88. 

(3) Cost of the Handbook on Experimental Procedures was calculated as $S.54 each, 
based on duplication costs only. The cost of additional material provided to the 
subjects on collecting and charting data (@ $.70), seconds/decimal conversion 
finders (@ $<I2), and practice sheets for the application of the dec is ton -rules (@ 
$.40) were calculated separately since tiiose materials were not originally 
planned for inclusion in the Handbook. 

ik) Total trainer and teacher costs were based on the length of training time ond the 
hourly rate estimated for salaries. 

Follow-up Costs . Costs for three different types of follow-up provided by 
project staff were calculated separately. Cost estimation for follow-up provided 
through telephone conversations included preparation time for the trainer, and trainer 
plus teacher salary costs for length of the conversation ((preparation time for trainer x 
hourly rate) + (length of call x trainer hourly rate) + (length of call x teacher hourly 
rate)]. Cost estimation for follow-up provided through personal visits included 
preparation time for the trainer, trainer and teacher salary costs for the length of the 
meeting, ond the cost of any materials consumed during the follow-up [(preparation 
time by trainer x hourly rate) + (length of meeting x trainer hourly rate * number of 
subjects) + (length of meeting x teacher hourly rate x number of subjects) + material 
cost]. Cost estimation for follow-up provided through written communication 
included preparation time for the trainer, typing cost, and materials cost, 
[(preparation time x trainer hourly rate) + (no. of pages typed x $.50) + materials] . 

Data Summarization 



Information collected on the questionnaires was entered into a computer for 
summarization. Information from each item and each subject was coded individually. 
Data were entered via SOS and summarized through the System 1022 computer 
programs. Reliability checks of a randomly selected 10% of the subjects were made to 
determine reliability of data entry. Errors in data entry were found on Background 
Questionnaire data, so each entry was rechecked. A second check on Background 
Questionnaire data indicated 100% reliability. Reliability checks for the remaining 
Questionnaires also showed 100% accuracy. 



ERLC 



'The software SOS program utilized was prepared by the Brookings Institution 
Computer Center (Version 23). 

2 The software SYSTEM 1022 (Version 113) utilized is copyrighted by The Software 
House, I I0S Massachusetts Avenue, Cambridge, MA., 1979. 
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RESULTS 

Evaluation of Training 

Subjects were asked to rate the clarity of training on a Ltkert scale from 
Confusing" (I) to "very clear 1 * (5). All subjects v*na completed training responded. 
Mean ratings across sites are shown in Table 16. 5/jbjects who received individualized 
training from Site 4 generally rated all areas as clearer in presentation than subjects 
from either the small or large group training sessions. Subjects with previous training 
in Precision Teaching (N*2I) rated the training as less clear than did subjects without 
a Precision Teaching background (N=33X The overall rating was 4.3 for the first group 
and 4.5 for the second group. Therapists (N=I3) rated each individual presentation 
topic as less clear than did teachers (N?38)» although the overall rating was higher (4.6 
for the therapists and 4.5 for the teachers). 



Tcble 16 

SUBJECTS' RATINGS OF CLARITY OF TRAINING* 



Presentation Topic 



Site 2 (Lorge Group) 



Site 3 (Smoll Group) 



Site 4 (Individualized) 



All Subjects 



General purpose of 
decision -rule procedures 

Establishing instructional 
formats 

Collect date 

Chart data 

Draw lines-of-progress 
Use decision-rules 
Remediation strategies 
Overall 



4. 3 (N=30) 

4.3 (N=3I) 

4.5 <N=23) 

4.5 (N=I4) 

4.6 (N*3I) 

4.4 (N=3I) 
4.2 (N=3I) 

4.5 (N=30) 



4.5 (N=l 
3.9 (N=l 

4.2 (N=I 

4.6 (N=l 

4.0 (N=l 
4.4 (N=l 
3.9 (N=l 

4.1 (N=| 



5.0 (N=I2) 

4.7 (N=9) 

4.5 (Nsl2) 

4.7 (N=I2) 
5.0 (N=I2) 
4.9 (N=I2) 

4.8 (N^I2) 
4.8 (N=I2) 



4.5 (N=53) 

4.5 (N=5I) 

4.5 (N=46) 

4.6 (N=37) 
4.6 (N^54) 
4.5 (N=54) 
4.3 (N=54) 
4.5 'N=53) 
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Attitude Following Troining 

Subjects were osked to rote o series of items in order to determine their ottitude 
toward the experimental procedures following training. Subjects rated their general 
attitude toward the procedures on a five-point Likert scale from "I" (very unfavorable) 
to "9* (very favorable). Subjects who received individualized training provided more 
favorable ratings than subjects from other sites, but alt mean ratings were above 4.2, 
and the overall mean was 4.38 (Table 17). No subject rated their general attitude as 
less than "3". Subjects rated their opinion as to the usefulness of the procedures in 
general on a five-point scale from I foot at all useful) to 5 (very useful). The subjects 
from Site 2, the large group, rated the usefulness higher than any other site (4.6), but 
alt groups were similar (Table 17). No individual subject rated usefulness less than "3". 

Subjects were also asked to rate the applicability of the procedures to their 
teaching situation ("l"= not at all applicable; "5"s very applicable). The mean ratings 
on applicability to their own situation were lower in many areas than the mean ratings 
for general usefulness (Tcbte 17). Subjects from Site 2 (Large Group training) rated 
the procedures generally more applicable in their settings than did teachers from the 
other two sites. Several subjects rated the applicability of conducting daily 
instructional programs according to a consistent plan in their situations as "2". One 
person rated the applicability of charting data as "I", white all other individual subject 
rankings were "3" or above. 

The mean ratings of the therapists were lower in all areas than these of the 
teachers. Therapists felt that conducting instructional programs on a daily basis was 
less apolicable to their situation (therapists mean rating was 3.6, teacher's mean 
rating was 4.6). Two of the lowest mean ratings shown in Table 17 are for the 
applicability of conducting instructional programs. If therapist's scores are seporatrtd 
from those of the teochers, however. Site 2 teacher^ mean rating is 4.5, Site 3 
teacher's rating is 4.7, and Site 4, where there were no therapists, remains at 4.8. 

Teachers who reported previous training in Precision Teaching generally rated 
both their general attitude, the usefulness, and the applicability of the procedures 
higher than did subjects without such training. "Hie mean rating of general attitude of 
subjects with Precision Teaching training (N*2I) was 4.48, as compared with a mean of 
4.18 for the "no precision teaching" subjects (N=33). This difference in mean ratings 
held true for all areas except charting. Subjects with Precision Teaching training 
rated charting as less applicable in their situation (4.23) than did subjects without 
Precision Teaching (4.48). 

Subjects with assistance in their classrooms (N=45) found the procedures more 
useful and applicable to their situations than did teachers who did not have any 
assistance (N~8). 



Table 17 

ATTITUDE TOWARD PROCEDURES FOLLOWING TRAINING 



Presentation Topic 



Site 2 (Large Group) 



Site 3 (Small Group) 



Site 4 (Individualized) 



All Subjects 



General attitude toward 

procedures 
General usefulness of 

procedures 
Applicability to their 

situation 

Overall 

Conduct instructional 
programs 

Collect performance 
data 

Chart data 

Use decision-rules 

Use remedial strategies 



4.4 (N=3I) 

4. 6 (N = 3I) 

4.4 (N=3I) 
4. 2 (N=30) 

4. 5 (N=3I) 

4. 5 (N=3I) 

4. 5 (N=3I) 

4. 6 (N=3I) 



4.2 (N=I0) 

4. 5 (N=I1) 

4.1 (N=ll) 

4.1 (N=ll) 

4. 5 (N=ll) 

4. 4 (N=ll) 

a. 2 (N=ll) 

4.2 (N=ll) 



4. 6 (N=I2) 

fl. 5 (N=|2) 

a. 3 (N=|2) 

4. 8 (N=I2) 

4.8 (N.I 2) 

4. 5 (N=I2) 

fi. 8 (N=I2) 

4. 8 (N=I2) 



ft. 38 (N=53) 
fi. 56 (N=54) 

4.33 (N=54) 
fi. 32 (N=53) 

4.57 (n s 5a; 

4.46 (N=54) 
4.5* (N=54) 
4.55 (N*54) 



v*» 



ranked on a five-point Likert scale, "1"= very unfavorable to very favorable 
^ranked on a five-point Likert scale, "l"= very useful to "5"= not at all useful 
'ranked on a five-point Likert scale, "1"= not at all applicoble to "5"= very applicable. 
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Cost of Training 

Subjects who received no training other than receipt of the Handbook presented 
the lowest training costs ($6.6fysubject)* Among subjects who received direct training 
In use of the decision-rules, the subjects trained in a small group of nine at Site 3 had 
the lowest per subject cast, at $35J5 per subject* This difference was probably due to 
the difference in length of training time between those subjects and subjects at other 
sites, although the trainer-subject ratio is also important (e*g M the cost for a subjecf 
trained individually at Site 4 for L92 hours was $«.66). (See Table 18.) 



TOT 18 

TRAINING COSTS* 



Cost Item 


Site 1 




Site 2 


Site 3 




Site 4 




Training ratio 


none 


1:17 


1:14 


1:9 


1:1** 


1:4** 


1:2 


Training time in hours 


none 


4.0 


3.75 


2.0 


2.75 


2.34 


2.67 


Trainer cost per 
subject @ $8*85/hr 


none 


$ 3.64 


$ 3.92 


$ 2.69 


$ 24.34 


$ 8.15 


$ 14.02 


Subject salary for 
training @$l2.88/hr 
per subject 


none 


$ 51.52 


$ 48.30 


$ 25.76 


$ 34.65 


$ 30.14 


$ 34.39 


Handbook & training 
materials 


$ 6.89 


$ 6.99 


$ 6.99 


$ 6.75 


$ 8.10 


$ 8.10 


$ 8.10 


Cost Per Subject 


$ 6.89 


$ 62. 15 


$ 59.21 


$ 35.15 


$ 67.95 


$ 46.39 


$ 63.16 


No. subjects trained 


27 


17 


14 


II 2 


6 


4 


2 


TVhtnl tr/ii \P>T ^ rwct 
IvIUI U %jt rci CUE>I 

• 




$ 61.88 


$ 54.88 


$ 29.59 


$142.26 


$ 32.60 


$ 28.04 


Total subject salary 


none 


$ 875.84 


$676.20 


$283.36 


$207.90 


$120.56 


$ 68.78 


Total materials 


$186.03 


$ 118.83 


$ 97.86 


$ 74.25 


$ 48.60 


$ 32.40 


% 16.20 


Total per training ratia 


$!86.03 


$ 1056.55 


$828,94 


$386.65 


$398:76 


$!72,27 


$126=32 



♦excluding travel costs 
**median per subject 
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Follow-Up After Training 

Follow-up was provided to subjects following training (or following receipt of 
Handbook ) at their request. Twenty-six of the 81 subjects (32%) completing training 
requested follow-up information: 4 of 27 (|5%) at Site I (Handbook Only); IS of 31 
(58%) of Site 2 (Large Group); 2 of 11 (18%) of Site 3 (Small Group); and 2 of 12 (17%) 
of Site 4 (Individualized). 

Follow-^) for four subjects of Site I included seven telephone calls. Cost of 
these calls (excluding long distance charges) was $40.64. Follow-up to eighteen 
subjects of Site 2 included seven short, individual meetings following training sessions 
and twelve telephone calls, Cost of these follow-ups (excluding long distance and 
mileage charges) was $6l.jO. Follow-up to two subjects at Site 3 included two 
personal visits, at a cost of $14.34. Three phone calls to one subject and a visit to 
another subject at Site 4 were provided, at a cost of $6.14 (excluding mileage). 

Evaluation of Handbook 

Fifty of the 81 subjects returned the "Hondbook Questionnaire'' (62%), 63% of 
Site I subjects, 45% of Site 2 subjects, 73% from Site 3 and 92% from Site 4. Subjects 
rated the content of the Hondbook of Experimental Procedures on a scale from 11 1 11 
(Confusing) to "5' 1 (Very Clear), As a group, the subjects tended to rate the text 
descriptions of the stages of learning and the instructional strategies related to them 
as more understandable than the decision-rules and the technical information on their 
application (Table 19). Subjects from the small group training rated the Handbook 
higher overall than subjects at other sites. 

Therapists (N*5) spent a median of 2 hours reviewing the Handbook, o* did 
teachers (N=44), and the mean median rating for the two groups was about the same 
(4.4 and 4.36, respectively). Subjects with a background in Precision Teaching rated 
the Handbook as less clear (4.27, Ns32> than did subjects without such training (4.58, 
N?I8), although those with training hod a higher return rote (74% and 40%, 
respectively). Subject who rated the procedures as most applicable (5) in their 
situation were more 1 *ly to return Handbook Questionnaires (Nzl7, 71%) and rate the 
Handbook highe* (mfc^i median rating 4.47) than those who rated the procedures as 
less applicable (hM6, return rate 53% and mean median rating of 4.38). Similarly, 
those who reported a very favorable attitude toward the procedures were more likely 
to return the Handbook Questionnaire (N=I6, 70% return) and rate the Hondbook as 
clear (mean median rating of 4.5) than those whose reported attitude was less 
favorable (N=I7, return rate 53%, mean median rating of 4.35). 




EVALUATION OF THE HANDBOOK OF EXPERIMENTAL PROCEDURES* 



Content Site I Site 2 Site 3 Site 4 All Subjects 

Handbook Only Large Group Small Group Individualized 

How to use 4.29 4.38 4.75 3.82 4.29 

decision-rules N=I7 N=I3 N=8 N=l I N=49 

Rules for Rate & 3.67 4.29 4.29 3.64 3.94 

Quick Reference for N=I5 N=I4 N=7 N=l I N=47 
Use of Rules 

Rules for % Correct 3.0 4.29 4.71 — 3.97 

Quick Reference for N=I0 N=I4 N=7 N=3! 
Use of Rules 

Quick References for 4.J5 436 4.63 4.27 4.48 

Intervention Strategies N=!7 N=I4 N^8 N=l 1 N=50 

Technical information 3.47 A.07 4.38 3.8 3.86 

for application (draw- N= 1 7 N= 14 N=8 N= 1 0 N=49 
ing I ines-of-pr ogress 
determining variability, 
etc.) 

Text: Instructional 4.65 4.57 4.5 4.18 4.5 

Formats N=I7 rt=l4 N=8 N=ll N=50 

Text: Acquisition 4.12 4.5, 4.63 4.18 4.34 

N=!7 N=I4 N=8 N=ll N=S0 

Text: Fluency- 4.47 4.29 4.71 4.18 4.39 

Building N=I7 N=I4 N=7 N=l I N=49 

Text: Compliance 4.38 4.5 4.75 4.45 4.49 

N=I6 N=I4 N=8 N=ll N=49 

Mean Median Rating 4.29 4.5 4.63 4.18 4.38 

Time Spent Reviewing 2.75 hrs 2.5 hrs 1.75 hrs 2.00 hrs 2.00 hrs 

O Median & Ron ge l-IOhrs .25-8hrs .5-3hrs .25-10 hrs .25-10 hrs 

ERIC trn 
*AI1 items were rated on a Likert Scale from I (Confusing) to 5 (Very Clear) O 7 
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Adoption of Decision-Rules 

The return rate for the "Trial Period Questionnaire" was 75% (61 of 81 returned), 
with 63% from Site I* 52% from Site 2* and 100% from both Sites 3 and 4 returning 
this questionnaire. Of these 61 subjects, 44 (72%) agreed to adopt the decision-rules 
foratrialP*Mod. 

The seventeen subjects who returned the questionnaire and decided not to adopt 
the procedures gave various reasons for their decisions. 

(1) l\Rne subjects reported that thev thought the procedures would take too much 
time. These subjects reported (on the Background Questionnaire) spending an 
average of 2 hours per week planning. Subjects who agreed to odopt the 
procedures* on the other hand, spent a mean of 4.6 hours per week planning. Two 
of these "no" subjects had no help during the week* and one subject had 6 hours 
of assistance per week. The five other ♦Yio" subjects had about the same amount 
of help per week (41 hours) as did the "yes" subjects who reported having 
assistance. 

(2) One subject reported that the procedures were not applicable to her severely and 
profoundly handicapped pupils since "ht<#i response frequencies* 1 were required 
for their implementation* and her pupils were not capable of such high rates 
(misinterpretation of procedures). 

(3) Three subjects reported that the procedure! were not corrpatible with their 
curricula. 

(4) One subject reported that the procedures were not compatible with the type of 
instruction provided in his/her class. This subject conducted programs once per 
week. 

(5) One subject* who also reported insufficient time to use the procedures* said that 
he/she was unclear how to apply the procedures* although this subject rated all 
areas as either clear or very clear following training* and did not request any 
follow-up. 

(6) Three subjects gave personal reasons (i«e.* unemployed) as reasons lor not using 
the procedures. 

The seventeen "no" subjects included ten teachers* I administrator and six therapists* 
while three therapists and 41 teachers agreed to use the procedures. Sixty~et<^t 
percent of the 'yes" subjects and 76% af the "no" subjects reported that other teachers 
in their school were using the decision-rules* while 87% of the **yes" subjects and 76% 
of the "no" subjects reported that their administration supported the use of the 
decision-rules (Table 20). A comparison of the two groups of subjects on a number of 
other variables is shown in Table 20. 
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Toble 20 

COMPARISON OF SUBJECTS AGREEING TO ADOPT PROCEDURES 
VS. SUBJECTS NOT AGREEING 



Variable 


Agree To Use (N=44) 


Not Agree to Use(N=l7) 


Av. Closs Size' 


7.46 pupils 


8*88 pupils 


Subjects with assistance 
Mean hours per wer '< 


91% 

41 hours 


76% 
38 hours 


Teaching experience 


3 years 


4.9 years 


Precision Teaching training 


36% of subjects 


4 1 % of subjects 


Conduct instruction at least 
once per day 


80% of subjects 


45% of subjects 


Provide at least 10 response 
opportunities 


59% of subjects 


24% of subjects 


Collect data on ot least 
85% of programs 


82% of subjects 


41% of subjects 


Collect dato every time 
progrom run 


64% of subjects 


18% of subjects 


Use rote dato 
ronked 0 \ dato type 


73% of subjects 
41% of subjects 


59% of subjects 
18% of subjects 


Use percent correct data 
ronked 0\ dato type 


68% of subjects 
43% of subjects 


7!% of subjects 
19% of subjects 


Graph pupil data 


61% of subjects 


41% of subjects 


Mean Planning time 


4.6 hours per week 


3.00 hours per week 


Mean ranking of training 


4.52 


4.5 


Mean ranking of genera! attitude 


4.55 


4.18 


Mean ranking of usefulness 
af procedures 


4.55 


4.6 


Mean ronking af applicability 
of procedures 


4.44 


3.8 



Excluding those with classes over 20 pupils per doy (7 teachers in "yes 11 column). 
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Actual Adoption of Procedures 

Of the 44 subjects who ogreed to use the procedures, 31 reported on the use of 
those procedures. The other thirteen subjects either failed to use the procedures or 
failed to report on their use. A comparison of those subjects who actually used the 
procedures (N=3I) ond those subjects who did not use the procedures (Ns50) is shown in 
Table 21. The subjects who did not use the procedures included all subjects who 
completed training, including both those who did not return a Trial Period 
Questionnaire (N=20), those who returned a questionnaire but decided not to use the 
procedures (N=*I7), and those who ogreed to use the procedures, but either did not do 
so, or did not report on their use (N=l3). 

Subjects trained individually (Site 4) showed the highest percentage of odoption, 
while subjects trained in the large group (Site 2) had the lowest rate (Table 22). 
However* the single most important variable in odoption may be previous training in 
Precision Teaching, ond only 19% of the subjects from Site 2 reported such training. 
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Table 21 

COMPARISON OF SUBJECTS USING DECISION-RULES" AND 
SUBJECTS NOT USING DECISION-RULES 2 



Variable 



Used Decision Rules 
(N=3l) 



Did Not Use 
(N=50) 



Pupils 

Totol pupils served 

% teaming disabled 

% mildly handicopped 

% moderotely handicopped 

% severely handicopped 

% profoundly handicopped 

% other 

% moderote + severe + profound 
Assistance 

subjects with help 
overage hours per week 
ronge 

Experience 

subjects with experience 
years of teaching 
range 

speciol educotian years 

% of subjects with Precisian 
Teaching troining 

Profession 

teachers 

theropists 

(administrotors) 



343 

3% of pupils 
10% 
16% 
41% 
17% 
12% 

74% 



93% of subjects 
49.4 hours 
4-122 hours 



97% of subjects 
6.03 years 
1-22 years 

5.53 years 



65% of subjects 



97% of subjects 
3% 
0% 



1234 

3% of pupils 

8% 
24% 
27% 
24% 
10% 

74% 



82% of subjects 
53 hours 
4-476 hours 



98% of subjects 
5.57 years 
1-28 years 

4.52 years 



40% of subjects 



68% of subjects 
26% 
5% 



'Subjects who agreed to use procedures and reported an their use (N=3l). 
2 

Includes subjects who did not return questionnaire (N=20)» 

subjects who did not agree to use procedures (N*17) and 

subjects who agreed to use but did not use or report an use (N= 1 3). 
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T ype of Instruction 



Used Decision Rules 
(N*3I) 



Did Not Use 

(NrSO) 



Meon Percentoge of Instructional 
Time 

in requested Individual pupil 
responses in 1: 1 settings 

jn requesting ;rvjividuol pupii 
responses ot natural occosions 

in requesting individual pupil 
responses In group settings 

in requesting unison responding 
in group settings 

Type of Instruction Used Most Often 

Percentoge of subjects who spenH more 
than 50% of time: 

in l:| settings 

ot natural occosions 

in group settings 

in unison responding 

Subjects who mix types of instruction 

Frequency of Instruction 

natural occasions, daily 

twice o day 

daily 

four per week 
three per week 
two per week 
one per week 



46% of instruction 
29% of instruction 
27% of instruction 
3% of instruction 



48% of subjects 
1 9% of subjects 
13% of subjects 
0% of subjects 
1 9% of subiects 



3% of subjects 
13% 
68% 

6% 

6% 

3% 

0% 



Number of Pupil Response Opportunities During Instruction 



1-5 trials 
5* 9 trials 
10-15 trials 
16*20 trials 
20+ trials 



29% of subjects 
10% 
42% 
13% 
6% 



53% of instruction 
22% of instruction 
18% of instruction 
5% of instruction 



54% of subjects 
18% of subjects 
1 1% of subjects 
2% of subjects 
1 1% of subjects 



4% of subjects 
14% 
42% 

4% 
16% 
12% 

9% 



25% of subjects 

21% 

32% 

5% 

7% 



10 or more 



61% 



44% 



48 



Plonning 



mean hours per week 
range 



Percentage of Programs Collect Data 

95-100% of programs 

85-94% 

75-84% 

less than 75% 

Frequency of Collection Per Program 

each time conduct program 

almost every time 

less 

Data Collection During Program 
every trial 

all trials in time period 
first and/or tost trial only 

Type of Data Collected * 

count of behaviors 
count of trials 
levels of assistance 
accuracy (% correct) 
rate 

other time data 
miscellaneous 



4.06 hours 
1-10 hours 



52% of subjects 
23% 
6% 
19% 



58% of subjects 
35% 
6% 



35% of subjects 

13% 

16% 



55% of subjects 

32% 

45% 

71% 

71% 

65% 

I0 ( ,i 



•Subjects commonly collect more than one type of dotn. 
Type of Data Most Often Used 



count of behaviors 
count of trials 
levels of assistance 
accuracy (% correct) 
rate 

other time data 



1 3% of subjects 
10% 

3% 
42% 
42% 

0% 



4. 1 8 hours 
1-20 hours 



42% of subjects 
21% 
5% 
28% 



39% of subjects 

37% 

23% 



52% of subjects 
37% 
7% 



58% of subjects 
47% 
60% 
61% 
63% 
47% 
4% 



25% ot subjects 

7% 

7% 
35% 
23% 

0% 



Graph Data 



68% of subjects 



40% of subjects 




Rules Avaikible 
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when to st^p ahead— all programs 


58% of subjects 


47% of subjects 


some programs 


42% 


44% 


when pupil met aim— alt programs 


71% 


46% 


some programs 


16% 


42% 


when to step bock — atl programs 


30% 


33% 


some programs 


30% 


39% 


when ta change strategies— all programs 


6% 


18% 


some programs 


47% 


26% 


what strategies to change— all programs 


10% 


14% 


some programs 


29% 


30% 


subjects with some rules 


100% 


9ti% 


lUihLpffc with ofl 


35% 


j/ to 


Rating of Training 


N,:23 


N=3I 


Overall 


4.52 


4.47 


general purpose of procedures 


4.7 


4.3 


data collection 


4,5 


4.3 


charting data 


4.7 


4.5 


draw I ines-of-pr ogress 


4.65 


4.52 


use decision -rules 


4.65 


4.39 



Attitude Following Training 


N=2J 


N=3I 


general attitude toward procedures 


4.57 


4.23 


usefulness of procedures 


4.61 


4.52 


applicability to their situation 


4.52 




overall 


4.19 


conduct instructional programs 


4.74 


4.0 


collect performance data 


4.83 


4.39 


chart performance data 


4.83 


4.19 


use decision -rules 


4.78 


4.36 


strategies 


4.74 


4.42 



Reque st Help Following Training 52% of subjects 68% of subjects 
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Support N^3I N*5C 

odmlnistrotive support use of 26% of subjects 40% of subject 
procedures 

other teachers use procedures 90% of subjects 53% of subject 



Rating of Handbook 


N=30 


N=20 


how to apply rules 


4. 2 


4. 5 


rules 


4.0 


3.8 


quick reference guides 


4.5 


4.5 


technical content 


3.7 


4.1 


establish instructional format 


4. 5 


4.6 


acquisition strategies 


4. 3 


4.5 


fluency-building strategies 


4.4 


4.5 


strategies for compliance 


4. 7 


4.3 


median rating 


4.4 


4.35 


time spent reviewing 


2.86 hours 


2.5 hours 
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Table 22 

RATE OF ADOPTION OF PROCEDURES BY TRAINING MODEL 

Handbook Only Large Group Small Group Individualized 

Subjects trained N=27 N*3I N=1I N*12 

Subjects with previous 
Precision Teaching 

Training 78% 19% 82% 67% 

Rate af return 1 67% 58% 100% 100% 

Agreed ta use 

procedures 56% 35% 55% 100% 

Actually used 

procedures* 30% 1?* 55% 92% 

Continue to * 

use procedures 87% 100% 83% 100% 

^Return rate of "Trial Period Question* jire", in which the subjects reported their decision to 
^Hopt or not adopt the procedures. Subjects wha did r * return the questionnaire are classified 
as "not adopting" 

2 

Includes only those subjects wha reported on use af the procedures. Calculated from the fatal 
subjects trained. 
3 

Those subjects wha reported that they planned on continuing ta use the procedures following 
the conclusion of the project. Calculated from those subjects who acutally used the procedures. 
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Application of Decision Rules 

The thirty-one subjects who reported on the use of the decision-rules applied the 
procedures in a total of 227 Instructional programs with 82 pupils over a total of 391 
weeks (Table 23). The age range of these pupils was 1-29* with the majority between 
3-23. Most applied the procedures with a few af their pupils* although some applied 
decision-rules ta at least one program for each pupil in their classroom. The highest 
proportion af pupils affected by t k » decision-rules were those taught by Site 2 
teachers, who were trained in the kuge group (Table 23), while teachers who were 
trained individually applied the procedures ta the fewest numbers of pupils. 

Table 23 

APPLICATION OF PROCEDURES 





Handbook 


Large Group 
Training 


Small Group 
Training 


Individualized 
Training 


Total 


Subjects using 
decision-rules 


8 


6 


6 


11 


31 


Total pupils served 


52 


71 


W1 


133 


303 


Pupils offected by 
^^cisionwules 


13 

25% 


30 
42% 


17 

36% 


22 
17% 


82 
27% 


Programs 


49 


95 


52 


31 


227 


Total weeks 
Mean per subject 


69 
8.6 


1 18 
19.6 


107 
17.8 


97 
8.8 


391 
12.6 



Subjects reported on the use af rare date in 72 programs, the use af adjusted rate data 
in W programs, the use of percent correct data in *2 programs, the use af duration per 
trial data in 30 programs, t ! <e use af duration per session in three programs and the use 
af latency data in six programs. Data types for the other thirty programs were not 
provided. Prior ta training, thirteen af the thirty-one subjects (42%) stated that they 
most often used percent correct data. Of these, two collected percent correct data 
exclusively for the trial programs, eight collected time-based data only, and three 
collected a mixture af chta types. Before training, eleven af the subjects (35%) stated 
that they mast often used rate uata, and these subjects applied the decision-rules in 
programs in which they collected time-based data. Six subjects reported that they 
preferred ta use either a count af behaviors or a count af trials data type, but, during 
the trial period, they applied the decision-rules primarily in programs In which time- 
based data were collected. 

The majority af subjects applied the minimum 'celeratton procedure far use af 
the decision-rules (Table 24), although six subjects from Sites I and 2 who were taught 
both procedures also used the draw patterns procedure. 
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Table 24 



APPLICATION OF DECISION-RULE PROCEDURES 



Procedure Number of Subjects Using 

Minimum 'celeration procedure for time-based 

data applied to time-based data 23 
Minimum 'celeration procedure for time-based 

data applied to percent correct data 3* 
Draw patterns procedure for time-based 

data applied to time-based data 3 
Draw patterns procedure for percent-correct 

data applied to percent correct data 3* 



*One subject tried both procedures 




Use of Handbook 



Eleven subjects rated the Handbook as less clear during application of the 
procedures than prior to application, five subjects rated the Hancfoook higher during 
app'ication, and thirteen subjects rated no change. The mean median rating of the 
Handbook prior to application was 4.48 and during application (on the first procedures 
Questionnaire) was 4.23 (Table 25). However, since the questionnaires differed, a 
comparison is not too useful, although the same scale was u*»d. The questions on the 
''Procedures Questionnaire" asked subjects to rate individual parts of the technical 
information section separately, although these topics had been grouped under one 
heading on the "Handbook Questionnaire". This change was made in order to 
determine which areas of the Handbook needed major revisions. 
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Table 25 

SUBJECTS' RATINGS* OF THE HANDBOOK DURING APPLICATION OF PROCEDURES 

Content First Report Second Report Third Report Fourth Report 

N=5 N=1 

How to use decision-rules 4.48 4.0 4.6 5 

N=29 N=ll 

Decision Rules and Quick- 4.58 4.64 4.8 4 

References Guides N-29 N= 1 1 

Technical Information 

Drawing Lines of Progress 3.7 4.09 4.4 5 

N=30 N=l I 

. Determining Percent Correct 3.2 3.6 5 

N=t0 

Determining Performance 3.31 3.73 3.6 5 

variability N=25 N^ll 

Sharp Deceleration in 3.62 4.2 4.2 5 

corrects N=2? N=I0 

Textt Instructional 43 4.18 4.2 4 

formats N=30 N=lt 

Tex+: Acquisition 4.17 4.0 4.0 4 

N=29 N=ll 

Text: Fluency-Building 4.07 4.0 4.4 4 

N=30 N=ll 

Text: Complice 4.47 4.46 4.8 4 

N=30 N=ll 

Mean Median Rating 4.23 4.18 4.2 4.5 

N=30 N=ll 

"Rated an a Ifcert scale of i (unclear) to 5 (very clear) 
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Attitude Tow or d the Procedures 

The attitude of the subjects toward the use of the decision-rules and the 
associated procedures was 'ower during application than prior to application (Table 
26). Subjects rated their attitude toward the collection of performance data as most 
favorable, and t3ward drawing lines of progress as most unfavorable overall. The 
subjects who reported more then once (N=l I) generally increased their attitucfe rating. 
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Table 26 

SUBJECTS' RATINGS* OF THEIR ATTITUDE DURING APPLICATION OF PROCEDURES 



Attitude Area 



Using procedures *hen 
programs are established 

Collecting direct 
performance data 

Charting performance data 



Drawing minimum 
'celerntion lines 

Drawing I ines-of -progress 



Using decision-rules 



Using intervention 
strategies 

Overall 



Prior to 
Application 



4.72 
4.81 
4.86 



During Application 



4.77 
4.72 
4.5? 



First Report 



4.14 

N=2? 

4.73 
N=30 

4.43 
N=30 

4.28 
N=30 

3.7? 
N=2? 

4.52 
N=2? 

4.48 

N=2? 

4.36 
M=30 



Second Report 
N=II 

4.45 



4.55 

4.36 

4.45 

3.?) 

4.36 

4.0 

4.0? 



Third Report 
N=5 

4.6 



5.0 
5.0 
5.0 
4.2 
4.4 
4.4 
5.0 



Fourth Report 
N=l 



5- 

4 

4 

4 

5 

4 

5 



♦Rated on a Likert scale of ! (unfavorable) to 5 (very favorable) 
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Flooning Time 

Prior to application J the procedures, the thirty-one subjects reported a mean 
planning time of 4.06 hours per week (range I- 10 hours per week). During the 
application of procedures, the group averaged 2.24 hours per week (range 1-5 hours) 
during the first reporting period. Of the thirty-one subjects, one subject reported an 
increase of I hour per week in planning time during the application of the procedures, 
and three reported no change in planning time. Twenty-six subjects reported a 
deceleration in planning time: ei^ht reported decreases of one hour per week; five of 
two hours per week; five of three hocrs per week; four of foui hours per week, and 
four of five hours per week. 

Subjects were asked to estimate how their picking time during the use of the 
procedures compared with planning time prior to use. The subject that reported an 
increase also estimated an increase, and of the three subjects whose planning time 
remained the same, one estimated an increase, one a decrease and one correctly 
estimated no change. Of the 26 subjects whose planning time octually decreased, only 
three correctly estimated the decrease, while 10 estimated no change and 13 
estimated an increase. 

Eleven subjects filed a second report. The median hours planning for this group 
prior to application was 3 hours per week (range UIO), at the first report it was I hour 
(range 0-5 hours) and, at the second report it was olso one hojr per week. Eight 
subjects reported no change between the first and second reports, two subjects 
increased one hour each, and one subject's planning time decreased by four hours per 



weeK. 



Five subjects reported a third time. Two of these subjects reported no change in 
their planning time between the second and third reporting period, two subjects each 
reported on increase of one hour per week, and one subject reported an increase from 
0 hours to two hours per week. 
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Impoct on Pupil Performance 

The impoct of the use of decision-rules on pupil performance was determined by 
subjects' estimation af impact, and by analyses of pupil performance data shared with 
the project staff. 

Subjects' Estimation af Impoct 

Twenty subjects, who applied the procedures in programs for 63 pupils, estimated 
that the use of the decision-rules accelerated pupil progress (Table 27), while seven 
subjects, with 12% of the pupi's, estimated that they made no difference to the 
progress of the pupils. One subject, who applied the procedures with three pupils 
estimated that pupil progress was decelerated. Three subjects, working with six 
pupils, estimated that the decision-rules accelerated progress in some cases, but 
decelerated progress in others. 

Table 27 

SUBJECTS' ESTIMATION OF IMPACT OF PROCEDURES ON PUPIL PROGRESS 



Estimation Subjects Pupils 

accelerated pupil progress 65% of subjects 77% of pupils 

20/31 63/82 

made no difference ta pupil progress 23% 12% 

7/31 10/82 

decelerated pupil progress 3% k% 

1/31 3/82 

accelerated in some, decelerated 
in same '0% 7% 

3/31 6/82 



Analyses of Pupil Perforraonce Data 

Subjects wha used the procedures were asked ta share pupil performance data 
with the project. Nineteen of ti thirty-one subjects shared data for a totcJ af 51 
pupils (62% af those affected by \ * procedures). Data for a total of 136 programs, 
60% of the total, were provided (Tou'e 28). All subjects from Site 3, the small group 
training, shared almost all of their data, while five of the eleven Site 4 subjects, wha 
were trained individually, shared little more than half of their data (Table 28). 

The charted data provided by the subjects were analyzed in order ta determine 
problems subjects encountered during application of the procedures. The daia 
provided by six of the nineteen subjects showed consistent errors in the application of 
the decision-rule procedures. These errors were of three major types: (I) failure ta 
change when pupil did not meet the minimum 'celeration line for three consecutive 
days; (2) failure ta correctly draw a new minimum 'celeration line after a change was 
node; (3) failure to draw lints-ol -progress. Subjects who made serious errors usually 
made all three typer of errors. Subjects who failed to change usually either charged 
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lote or drew o new minimc-n 'celerotion tine insteod of changing o strategy. Most tote 
changes, and oil coses where new tines were drown, foiled to accelerate pupil progress. 

Pupil performance dato, including those in vhich subjects foiled to correctly 
opply procedures, were anolyzed to determine the types of intervention strategies 
selected and their impact on pupil performance. An intervention was categorized as 
successful if it resulted in (I) an immediote step-up (increase) in the level of correct 
responding greater than xl.09 ; or, (2) accelerating correct performance of ot least 
xl.09 if previous performance was unchanging or decelerating, or, if previous 
performance was accelerating, o xl.09 improvement in acceleration; or, (3) in the case 
of prror correction procedures on immediote step-down (decrease) in the level of error 
responding greater than xl.09 ond o deceleration of error responding of ot teost 41.09. 

Each Intervention anolyzed had o minimum of five data doys both before ond 
offer the intervention. In cases where changes were step aheads in the curriculum, the 
success or foilure of the change was not determi'«ed, since the results of such on 
analyses would depend in great measure on the suitability of the curriculor steps and 
an the performance oims selected by classroom teachers. A summory of the pupil 
performance dato ic shown ?n Table 29. More detoiled descriptions of the doto are 
included in the individual site reports. 

The success of the subjects' decisions to change instructional strategies ranged 
from 0% (when one change wos mode) to 100% (when nine changes were mode). As o 
groqp, the nineteen subjects who shared dato successfully remediated pupil progress 
68% of the time remediotion changes were mode. Subjects from Site 2 were generolly 
more successful than subjects from other sites. Subjects from Site 4 showed the 
lowest percent of success, 59%. 

The strategies subjects actuolly implemented were compared with the strategies 
that were defined for each case through oppMcotion of the decision -rules, to determine 
hew often teachers actually implemented the suggested remediotion strategies (Toble 
30). Teachers could, for exomple, apply the decision-rule procedures ond determine 
that the rules suggested an acquisition problem, ond that some strategy designed to 
provide information on how to respond should be implemented. The teacher could, 
however, choose instead to add roisins for correct responses. In such o cose, the 
teacher would not be following o recommended strotegy. 

Subjects who generolly used o recommerJed strotegy (e.g., subject 5207) had 
higher success rotes than subjects who used recommended strategies less often (e.g., 
subject 5404), olthoagh subject 5339 followed the rules four times, ond pupil 
performance improved only twice. Overoll, the subjects applied recommended 
strategies 60% of the time. Subjects from Site 2 opplied the recommended strategies 
most often, and olso had the highest percentage of successful decisions. 

Pupil progress moy be acceleroted by use of the decision-rules under two 
conditions: (I) the pupil proceeds more quickly through the curriculum, taking fewer 
days to reach oim than without the use of the rules or (2) the frequency of remediotion 
decisions decelerates or (3) the percentage of successful remediations accelerot *. 
Borh (2) and (3) obove should olso result in foster progress through the curriculum. 



1.09 was selected since it is the quantity of change that con be generally noted by 
isuol inspection of the dato; although oil onolyses were made using standard trend 
ofcufotian procedures. 



FRIf^ visual mspecTion of Tr 
££yjbi colculotian procedures 
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The pupil performance dato were analysed in order to determine the possible 
impact of the procedures. Table 3 1 shows the percentoge and number of step oheods, 
which show progress through \he teachers' curriculum, ond the percentoge of 
successful remediotion changes olongside the subjects' awn estimation of the impact of 
the procedures. Four of the subjects who provided dato on five pupils showed that in 
265 days in thirteen programs their pupils did not advance o single curricular step. 
This includes 474 calendar days. During this time, only 8 changes were made in order 
to remediote performance. Those subjects esttmoted that pupil progress wo$ 
occeleroted. Those subjects either set extremely slow rates of acceptable 
acceleration for pupil performonce, or else they foiled to change strategies when pupil 
performance fell below the minimum 'celerotion line. 

Ten subjects with sixteen pupils made fewer than 10 changes each in their 
progroms. Eight of these subjects estimated that pupil progress occeleroted, including 
two subjects who did not make any changes. The rote of successful decisions for 4 of 
those 10 subjects cannot be determined, since they made no remediotion chonges. 
Three other subjects (5346, 5402 ond 5502) mode one decision each. 

Nine subjects made more than ten changes each. The success rote for those 
subjects ranged from 33%* 1 00%, with o median of 68%. 

Estimotians of the impact of the decision-rules connot be checked in the 
majority of coses, since doto on pupit performance both before and during the 
application of the decision-rules were not made avoilable to the project. Data 
avoitabte from three subjects ond on aide of one of the subjects does show pupil 
performance both before aid during the opplicotian of the decision-rules (Table 31). 
These data suggest that the major impact of the decision-rules wo^ to increase the 
rvjmber of changes, especially chonges in instructional strotegies for remediation of 
problems. However, the actuol impact of the procedures on pupil performance cannot 
be astermined by the data provided. 

Accuracy of Decision-Rules 

Notations mode by the subjects of the types of intervention strotegy changes 
they made were used to determine whether or not the chonge wos in accord with the 
intervention strotegies suggested by the rules. A change made in accord with the rule 
is a de focto prediction that the strategy will occelerate pupil progress. The number 
of times this prediction holds true ond pupil progress does acce!erote describes port of 
the predictive validity of ihe decision-rules. This prediction held true for 74 of 86 
cases, using the definition of a successful impact described in the previous section 
(Table 33). Converselv, a decision to use a nonrecommended strategy would result in o 
predicrian of pupil foilure; again providing the opportunity to determine the predictive 
volidity of the rules. The prediction of faMure held true for 37 of 57 cases. Overall, 
the predictive validity of the rules held for 1 1 1 of 143 cases, or 77.6% of the time. 
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Table 28 

SUBJECTS' APPLICATION OF PROCEDURES 





Handbook 


Large Group 


Small Group 


Individual 


. Total 




Only 


Training 


Training 


Training 




% Pupils for whom 


25% 


42% 


36% 


17% 


27% 


procedures were applied 


N=52 


N=7I 


N=47 


N=I33 


N=303 


Programs 


49 


95 


52 


31 


227 


Weeks 


69 


118 


107 


97 


391 


per subject 


8.6 


9.6 


17.8 


8.8 


12.6 
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Table 29 

SUMMARY OF PUPIL PERFORMANCE DATA 



Subjects who shared 
data 

Pupils on whom data 
made available 

Programs an which 
data made available 

Calendar days 
covered by graph 

Data days an graph 
Phases 

Total changes 

^% step aheads 

% remediation 

Analysed Remediation 
Changes 

% Successful remediation 
Changes af total 
remediation changes 

% Using suggested 
strategy af total 
remediation changes 

Predictability af 
decision -rules 



Handbook 
Only 

63% 
N=8 

69% 
N=l3 

63% 
N=49 

1757 



739 
102 
71 

18% 
82% 

34 

71% 

58% 

65% 



Large Group 
Training 

50% 
N=6 

63% 

Nr30 

51% 
3782 



481 

140 
92 
59% 
41% 

35 

86% 

84% 

77% 



Small Group 
Training 

100% 
N=6 

88% 
N=I7 

87% 
N=52 

4690 



2319 

336 

295 
57% 
43% 

79 

71% 

55% 

85% 



Individual 
Training 

46% 
N=ll 

36% 
N=22 

55% 
N=22 

645 



281 
38 
26 
69* 
31% 

7 

59% 
57% 
71% 



Total 
61% 

N=3I 

62% 
N=82 

60% 
N=227 

10,874 

3,820 
616 
484 
52% 
48% 

155 

68% 

60% 

78% 
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Table 30 

COMPARISON OF SUCCESS WITH USE OF SUGGESTED STRATECIES 



Subject Pupils Remediation % Successful % Interventions Using Predictability 
Interventions Interventions Suggested Strategies af Rules 

s;?e I 

5302 2 22 68% 53% 60% 

5321 4 6 83% 100% 83% 

5339 I 4 50% 100% 50% 

5346 I I 100% 100% ICO-s 

5347 10- 

Site 2 

5 101 6 9 100% 60% 60% 

5108 7 16 69% 65% 75% 

5207 6 II 91% 91% 82% 

S te 3 



5402 


1 


1 


0% 


0% 


100%, 


5403 


1 


6 


33% 


50% 


88% 


5404 


2 


12 


58% 


55% 


82% 


5405 


5 


27 


56% 


56% 


92% 


5413 


1 


0 








5414 


3 


48 


61% 


41% 


8?% 


aide ta 












5414 


4 


10 


90% 


90% 


80% 



Site 4 

5502 2 1 0% 0% 100% 

5503 2 3 100% 66% 55% 

5504 10 

5508 I 3 66% 66% 66% 

5512 2 0 
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Table 31 

PERCENT OF CHANGES IN CURRICULAR LEVEL 
AND IN INSTRUCTIONAL STRATEGIES AND SUBJECTS' ESTIMATION OF IMPACT 



Subject 


Pupils 


Programs 


Data 


Number 


Percent 


Percent 


Percent 


Subjects EstifTwtion 




Days 


Changes 


Step Aheads 


Remediation 


Successful 


of Impact 


Site 1 


















5302 


2 


15 


668 


42 


31% 


69% 


68% 


Accelerated pupil progress 


5321 


4 


6 


134 


7 


14% 


86% 


83% 


Accelerated pupil progress 


533? 


1 


4 


122 


6 


0% 


100% 


50% 


Accelerated pupil progress 


5346 


1 


4 


55 


1 


0% 


100% 


100% 


Accelerated pupil progress 


5347 


1 


2 


29 


0 


— 


— 


— 


Accelerated pupil progress 


Site 2 












61% 


100% 


Accelerated pupit progress 


5101 


6 


II 


413 


18 


39% 


5108 


7 


25 


838 


43 


63% 


37% 


69% 


Accelerated pupil progress 


5207 


6 


II 


310 


31 


65% 


35% 


91% 


Accelerated pupil progress 


Site 3 












100% 






5402 


1 


1 


18 


1 


0% 


0% 


No difference to pupil progress 


5403 


1 


3 


138 


14 


43% 


57% 


33% 


No difference in some, 
















accelerated in some 


5404 


2 


4 


237 


23 


13% 


87% 


58% 


Accelerated in some, 
















decelerated in some 




q 


12 


540 


74 


6!% 


39% 


56% 


Accelerated pupil proare&c 


5413 


1 


j 


62 


o 
O 


100% 


0% 




No difference to pupil progress 


5414 


2 


3 


513 


89 


46% 


54% 


61% 


Accelerated pupil progress 


5414 
















Accelerated pupil progress 


aide 


3 


14 


811 


86 


77% 


23% 


90% 


Site 4 
















Accelerated pupi 1 progress 


5502 


2 


2 


28 


3 


66% 


33% 


0% 


5:>03 


2 


6 


120 


17 


76% 


24% 


100% 


Accelerated pupil progress 


5504 


1 


1 


20 


2 


!00% 


0% 




Accelerated pupil progress 


5508 


1 


1 


72 


4 


20% 


80% 


66% 


Accelerated pupil progress 


5512 


; 2 


2 


41 


0 








Accel or a^ppupil progress 
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PUPIL PROGRESS IN A PROGRAM BEFORE AND DURING DECISICN-RULES 



FRir 







Before Use of Decision-Rules 






During Use of Decision-Rules 




Subject 


Pupil 


Av. Days to 
Step Ahead 


Remedial 
1 nter vent i ans 


Percent 

Success 


Av. Days to 
Step Ahead 


Remedial 
Intprvpn— 

tions 


Percent 

Success 


Percent 
Follow Rule 
Suggestion 


5321 


1 


38 


0 






, 


100% 


100% 




i 




o 




27 


( 


100% 


100% 




2 


31 


o 






! 


0% 


100% 




3 




o 






! 


100% 


100% 




4 


41 


o 






( 


100% 


100% 




4 




o 






! 


( \J\J TO 


100% 


5403 






o 




43 




0% 


0% 




, 




o 




14 


2 


100% 


100% 




1 


1 1 


o 




24 


2 


100% 


100% 


5414 




21 


2 


100% 


8 


2 


100% 


100% 










0% 


17 


u 


66% 


33% 






5 


0 




9 


5 


75% 


75% 






17 


8 


33% 


16 


6 


33% 


33% 




2 




| 


100% 


7 


7 


80% 


80% 




2 




2 


0% 


27 




100% 


50% 




2 


10 


0 




35 


2 


0% 


0% 


414 


I 


22 


I 


100% 


8 


3 


:oo% 


100% 




I 




1 
i 


100% 


1? 


o 








1 


24 


i 


0% 


13 


n 
u 








L 


17 


2 




1 7 


2 


50% 






2 




1 




23 


1 


100% 


100% 




2 


8 


2 


50% 


II 


2 


100% 


100% 




1 


5 


0 




9 


0 








3 


21 


2 


50% 


20 


2 


100% 


100% 
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Table 33 

PREDICTSVE VALIDITY OF DECISION-RULES (N=I44) 



Changes made in accord 
with a rule (prediction 
that progress will be 
accelerated) 

Changes made not in 
accord with a rule 
(prediction that progress 
will not be accelerated) 



Pupil Progress 
Actually Did 
Nat Accelerate 



Pplt Progress 

Actually 
Accelerated 



8.3% 


51. 7%* 


(N=)2) 


(N=74) 


25.8%* 


13.9% 


(N=37) 


(N:20) 



^Concurrence af prediction and actual results indicates prcaictive validity: 
OVERALL 77.6% (II 1/143). 



Continued Use of Procedures 

Twenty-nine af the thirty-one subjects who used the procedures reported that 
they plan on continuing ta use the decision-rules. One teacher wha did nat plan on 
using them was planning ta work with mildly handicapped pupils, rather than severely 
handicapped pupils. 



0 0 



67 



Cost Analyse 

Costs for the subjects who cpptied the procedures run from a median or $1145 
for subjects in the "Handbook Only" group to 574.24 for the median cost of a subject in 
the "Small Group Training" (Table 34). However, calculating costs per pupil puts the 
higner cost at $23.45 for the subjects trained individually, since they worked with the 
fewest numbers of pupils. The median costs per program aid per week are also highest 
for Site 4 subjects, who applied procedures for fewer programs. The n^dlan cost per 
program per week was lowest for Site I subjects, at 7£, arJ highest for S!t^ 4 subjects, 
at $4.69. However, costs will be decreased the longer the subject applies the 
procedures. 
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Table 34 

SUMMARY OF COSTS FOrt SUBJECTS APPLY INK" 



JRES 



Site I 
Hondbook 



Site 2 
Large Grou p 



No. of Subjects 

Median Training & Fol tow-up Costs 

Low 

High 

No. of Pupils 

Median Cost Per Pupil 

Low 

High 

No. of Programs 

Med" Cost Per Program 

Low 

High 

No. of Weeks 

Median Cost Per Week 

Low 

High 

Median Cost Per Program/Woek 

Low 

High 



8 

6.89 
>23.88 

13 




49 

$ !.59 
$ .69 
$11.18 

91 



> 66.87 
; 59.21 
! 1 37. 55 

30 




95 

$ 2.57 
< 1.91 
$ 34.71 

118 



1.33 


$ 4.08 


.63 


$ 2.73 


3.73 


$ 5.85 


.09 


$ .23 


.06 


$ .07 


1.86 


$ 2.04 



Site 3 
Small C^oud 




17 

$15.15 
$14.53 
$95.64 

53 

$ 6.05 
S 3.50 
575.85 

71 




.38 
^.46 
.18 



Site 4 
Individualized 



II 

68.54 
46.90 
1125.66 

22 

23.45 
20.94 
75.02 

31 

19.08 
12.45 
75.02 

97 

6.61 
5.71 
15.79 

4.69 
1. 10 
11.06 
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Precision Teachers 

In order ta implement the decision-rules in instructional programs, certain 
prerequisite conditions needed ta be met. It was hypothesized that teachers whose 
instructional and evaluation practices embodied the prerequisite conditions would be 
more likely ta odapt the procedures than those subject;, whose pre-training practices 
embodied none af the prerequisite conditions. Subjects whose instructional and 
evaluation practices indicated that many af the prerequisites for use of the decision- 
rules were identified from information provided on the Background Questionnaires. 
These subjects conducted daily instructional programs, in which the pupil was given a 
minimum of ten response opportunities, and in which data were collected on every 
trial. These subjects collected data on 95-100% of their Instructional programs, 
graphed their data, and used some decision-rules prior to participation in this study. 
The data collected by these subjects were usually time-based data: rate, latency or 
duration. 

Subjects whose instructional and evaluation practices indicated that mast of 
their current practices would need to be modified in order to use the decision-rules 
were also identified. Those subjects had no precision teaching training, conducted 
instructional programs at most three times per week, collected data approximately 
half the time they ran instructional programs, and preferred not to collect rate or 
other time-based data. These subjects also did not graph their data. 

Twenty-one subjects were identifed whose Instructional and evaluation practices 
were compatible with the use of the procedures. Of those, sixteen (76%) agreed ta 
odopt the procedures and eleven (52%) actually reported on the application of the 
procedures. Eight subjects were identified as those who would require major 
modification af their existing procedures In order to appiy the decision-ruks. Of 
those, only one returned a trial pericd questionnaire, and none agreed to try the 
procedures. 
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DISCUSSION 

We planned to evaluate the effectiveness af training models, as well as the 
effectiveness of the decision-rules, through an analyses af the impact af (I) the 
proportion of subjects from each site wha actuali/ adopted decision-rules, and (2) by 
the impact the rules had on pupil progress. Unfortunately, sever I factors prevented 
the cat lection af all af the data necessary for those evnluationsi the low return rate 
on questionnaires, especially during the odoption of procedures; the fact that teachers 
shared only about 60% af the pupil performance oat^ with Jhe project; and the lack af 
"before" or pre inter vent ion data on pupil per forme ;e. Each rijndition could have 
been improved through a more careful monogement af contingencies. The subjects 
were required ta complete training before they received a copy af the Handbook, and 
all buj 8% did complete training. Mast subjects we***, as expected, eager ta receive 
their copy, but, once they received their copy, participation dropped, and twenty-five 
pe .ent failed ta return the Trial Period Questionnaire. Participotlon continued ta 
drap, with 28% refusing ta try the procedures, anc 10% w^a agreed ta try did nat da 
sa, and finally, 29% did nat want ta shjre pupil data. 

The Handbook might have been used as a potential reinforcer more ef 'actively if 
subjects had "earned" sections af the Handbook throughout the project, instead af 
earning the entire book at the conclusion af trainhtg. Other available contingencies 
included: self -reinforcement by potential or actual improvement in pupil progress; 
reinforcement by discussion with research staff; self-reinforcement foi learning new 
techniques <ind facing new challenges; and reinforcement by pe r s and/or 
odministratars. Of those, only the potential reinforcer of "fallow-up'- could be 
controlled by the research staff. Additional information and assistc/tce in the use af 
the decision-rules, and assistonce in any after area requested bv the staff, was 
promised only for subjects agreeing ta employ the dpcision-^ules on a trial basis. That 
contingency was nat employed effectively at two sites. Three Site I subjects, in the 
New England area, adrait*> nonaged ta receive a total af six follow-up telephone calls 
by first ogreeing ta odapt the procedures, otkJ after the fajlow-up, nat using the 
procedures. They also received numerous phone calls requesting the return of 
"Procedures Questionnaires. Subjects from Site 2, the Summer Workshops, received 
fallow-up assistance from staff af CITPD on a noncontingent basis. CITPD*s faltaw-up 
included personal visits following the conclusion af the workshops, which the project's 
travel budget could naf afford. Eight subjects from those sites received sallow-up 
information without returning '^-ial Period Questionnaire". When called ta prompt a 
return of this questionnaire, those subjects asked for, and received, assistance an 
different programs in thdr classrooms. One su/ject ot Site 4 also managed to receiv- 
fallav'-up without actuoily trying the procedures. Perhaps better mat jement af the 
two contingencies available ta the research staff —the Handbook and fallaw-up~would 
have Improved the return rate on quest icmai res, the rate oi adoption of decision- 
rules, and the amount af pupil performers data shared with the project. Even with 
such Poor contingency management, however, thirty-one af the original eighty-one 
subjects remained with the project, and nineteen subjects shared pupil performance 
data. 

Effectiveness of Training Models 

There are several different methods af estimating the effectiveness of thr 
various training models. By far, trainee rating scales ore the most common. Site k 
subjects, who received individualized training, gave the highest ratings ta the training, 
while those subjects trained in a small group (Site 3) gave the lowest ratings. Since 
attitude toward the procedures is also important, it is interesting that subjects from 
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the individualized training model rated their attitudes as more favorable than :he 
ather si'es in general attitude and in the applicability af various components, but that 
subjects from the large group workshop training rated the procedures as slight!/ more 
useful and applicable. 

Another test af training is the amount af fallow-up required in order to 
implement the procedures, jhe better the training, the less fallow-up should be 
needed. Subjects from the "Handbook Only" group requeued the least amount of 
fallow-up* bath before and during application af the procedures, while all but one 
subject from bath the workshops and small croup training requested fallow-up. 
However, the data shared by participants indicates that perhaps 32% af all subjects 
applied the procedures incorrectly— more subjects should have requested technical 
assistance. Errors in application af the procedures were not remedied unless teachers 
asked for technical assistance. In most cases, assistance provided concerned 
instructional methods, rather than technical information on the application of the 
decision-rules. In cases where techn*'al information was requested, it was only 
through a discussion of same related topic that the teacher realized a problem in 
application. In the majority af cases, the subjects were unable to recognize an error, 
and so did not ask for assistance. Errors in application af the procedures could have 
been prevented not only by improving the training and H o r n book, but by requiring a 
technical fal low-up session for teochers adopting the procedures Of course, with so 
few contingencies available, such a requirement may have further reduced 
participation. 

Ideal ly, teacher training should produce a demonstrable impact on pupil 
performance. The first step In achieving that impact is the adoption of the pra edures 
by the teachers. The individualized training model produced the Highest rate af use in 
the classroom (92%) while the large group training producec the smallest (19%)* 
However, background variables such as previous training in Precision Teaching 
probably affected rate af adoption as much, if not more, than did the type af training 
the subjects received. The second step in such an evaluation is the number of pupi ls 
affected. Subjects from the large group training worked with the highest percentage 
af pupils (42%) while subjects from the individualized trairing worked w r th only 17% af 
their pupils* While a greater number af teachers may have agreed to try the rules in 
their classroom when individual training was provided, those some teachers actually 
tried the procedures with fewer children than those given less direct support initially. 
Mast subjects trying the rules agreed that the procedures cccelerated pt*pil progress, 
but the actual data required for a comparison af pupil progress before and during use 
af the decision-rules are not available. Overall, then, it is really impossible to 
determine which training model produced the greater impact on pupil performance. 

Results suggest that perhaps training in a large group as conducted during this 
project will require a great deal af fallow-up, and result in a law rate af adoption. 
Those that da adopt the procedures, however, will apply them io many of their 
students* Conversely, the project's small group training will produce a higher rate af 
adaption but affect fewer pupils. Individualized training will produce high ratings far 
training content, and high rates af adoption, but also affect only a small proportion of 
pupils. The "textbook only" model will require some follow-up, and produce moderate 
amounts of adoption that affect cbour half af the pupils in the classrooms involved. 

Independent Variables Affecting Adoption 

This study was not designed to test the relotive Importance af various 
independent variables in affecting rate af adoption. However, there are some major 
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differences between those subjects who adopted decision-rules and those who did not 
odapt them. Of primary inportance is the fact that 65% of those adopting decision- 
rules reported previous training in Precision Teoching, while only 40% of those that did 
not tse the decision-rules reported such training. That difference atsa shows up in 
related variables (e.g., frequency af instruction; number af pupil response 
opportunities; frequency af data collection, etc.). 

Of secondary interest, a greater proportion af the teachers adopted the 
procedures than did therapists. Half af the therapists included in the st^dy would have 
had ta modify their instructional and evaluation practices considerably for the 
application af decision-rules, but half af the therapists were already using precision 
teaching techniques. Only one therapist, who had na previous precision teaching 
training, actually adopted the procedures. That may have been a result af the 
training, since therapists generally rated the Individual topics as less clear than did 
> *Khers. In addition, their attitude toward the procedures fallowing the training was 
less favorable than the teachers. 

Subjects who requested additional information immediately fallowing training 
were less likely ta use decision-rules than those who did not reojest such help. On the 
other hand, there are several additional variables that may affect who asks for help 
and who does not ask. 

Ninety percent af those subjects odopting the procedures though* that other 
teachers in their areas were using the decision-sules, while only 53% af the 'Via** 
subjects thought so. Sl more af the "no" subjects thought they hod odministrative 
support than did the "yes" subjects. This may suggest that peer participation is more 
af a potential reinforcer than administrative support. 

ft training designed ta impact on tearhers' use af decision-rules were again 
provided, the rote af adaption or behavior s lange on the port af teachers might be 
accelerated by including those teachers wit previous training in PirecL m Teaching 
and by selecting teachers who work together. Modification in the training ta include 
more examples af applications in traditional therapy situations might improve adoption 
rate by therapists. 

A true cost-benefit analyses of the training is impossible, since one would have 
ta assign a monetary value ta improvement in ^jpil performance. Since-rwe are 
unwilling ta do so, we can only point out that the cost af the decision-rules may run as 
low as 7£ per week per program. Costs could be substantially reduced by increasing 
the rates af adaption and af application ta pupils and programs. 

Effectiveness of Decision-Rules 

The effectiveness af the decision-rules in accelerating pupil progress cannot be 
accurately gauged without data on pupil performance before the use af the procedures. 
Such data were not available ta the project. However, the percentage af successful 
decisions mode by the subjects participating in the project compares favorably with 
teachers who used the decision-rules in previous years. 

Overall, the propo n ion af changes designed ta advance the pupil through the 
curriculum was higher in the 5th year (Figure 5), but could, af course, be heavily 
influenced by the type af curriculum secfjencing used by teochers. The fifth year 
subjects made slightly fewer interventions designed ta remediate failing performance 
than in previous years, and they were much more successful in their decision making 
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than teachers who did not use decision-rules (Figure 6). All of the teachers who used 
decision-rules during the lost three years of the project hod a higher percentoge of 
successful decisions than teachers who did not use such rules. However the data 
indicate thot the decisions have a higher chance af success if rule suggested strategies 
are implemented (figure 7). 



The decision-rules developed during the five year \foject show great promise in 
improving the success of teacher decision-making in instructional progroms. The 
decision-rules ore designed ta facilitate pupil performance during the acquisition and 
fluency-building stoges af learning. Those stages of learning can be identified from 
charted records af pupil performance. Certain types of instructional strategies have a 
high probability of accelerating Seaming if appropriate!) implemented during each 
stage. Remediation strategies designed to provide information on how ta peform the 
desired skill or behavior are generally effective during acquisition* white consequences 
thot motivate performance are generally more effective during the fluency-building 
stage of learning. Decision-rules that can be applied ta pupil performance data may 
not only reduce the amount af planning time spent by a teacher* but will increase the 
effectiveness af teacher decisions. These result* hove the potential af improving the 
education af the severely handicapped. The decision-rules can be taught ta classroom 
teachers with a minimum af background skills in relatively short periods of time* from 
2 ta k hours. These results ore very encouraging* and indicate +he potential impact of 
the use af decision-ruies in improving the education of the severely handicapped. 

Other results* however* give cause for concern. Although not initially included* 
the decision-rules now include a rule saying thot if th& pupil is at aim* it is time to 
move to the next step. This rule had to Se included because mony teachers did not set 
aims for their programs* and/or did not move the pupil to the next step once on aim 
was reoched. 

Another cause for concern is the low rating given by fifth year teachers of the 
applicability af conducting doily instructional programs. Sufficient evidence hos 
accumulated on the need for frequent instruction to insure that skills are acquired* 
mastered and maintained. Why aren't teachers ready ta conduct doily programs? 

This project was not designed ta intervene in the curriculum offered by the 
teacher • However* the !>st of behaviors targeted for intervention has not changed too 
much over the years. The behaviors targeted by the fifth year teachers were simitar 
ta those targeted by teachers five years ago* ond the lists (provided in th*; site reports) 
do not indicate thot the progroms are teaching those behoviors that would promote 
independent - .nrnuntty or home functioning* despite the coll for such progroms in 
recert years. The teachers who participated in the study are weil trained and well- 
meaning* yet they continue ta teach skills that hove seemingly little relevance to 
community-based functioning. 

Another problem lies in the collection af data. Despite the fact that the 
collection of at least same data is mandated by P. L, 94-142* several teachers reported 
that they collect such data an fewer than half of their programs. Data collection and 
charting is often promoted as a routine task for teachers, rather than one that can 
decrease their planning time and improve pupil performance. Th* deve!opn*ent af 
datc-bo$ed decision-rules can help teachers improve the success of their instruction* 




shaded area = decisions to remediate oupM performance 
unshaded area =* decision to mov^ to next instruction step 
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Figure 6 

Successful Remediation Decisions Durinq Five Yaars 
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but we see little hope for the implementation at decision -rules until such time as 
teachers are willing and ready ta conduct instructional programs with specific 
performance aims and ta collect and chart their pupils' performance data* Even then* 
the full impact af decision-rules will nat be felt un*il they ore cppiied to programs 
designed tc teach functional skills ta the severely handicapped. 
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Figui^^ 

COMPARISON Or SUCCESSFUL DECISIONS WITH USE OF SUGGESTED STRATEGIES 



RE MEDIATION DECISIONS 

shaded area - improve pupil performance 
unshaded area - fail to Improve 
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unshaded area - use of other strategy 
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APPENDIX A 
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r her.iplat »f oer up<it ion* 1 th<T4pifi ts > enmtnini i .it I on d I ftt»rd<>r«] ape ( iniisK) 
provldi direct «<*rvJfe (Inatructlnn or th^xjipv* t>> pnc nr ranr* of /i.ut 
itndi nt * 

Q n " [ J'7Ej* *> fnr (*.»» averse tn*-if t*1 | "j funurft per wt-rfc 
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ho vi>n ^rrvf n*i >i "vtMvlvnr loi nn i * i> r % i t v « H it i» /M ph hi honl studnnt 
r iwti li< rs «n pr u t J ► uu ^.f ■>-!« tit <% * 

Q | J P'.t:a^> Avi t i+i* n ruirali* t Ijt ijtMtfft j 1 t„r I I total 

h j. j I I pfr w 



hour*i 
week 



PAH / HAfM K n *"*W 



I Niimhri of veiTfi te.n hi ni* h.n>l ^""j 

1 Number tvf ve.iri trichjn^ the bandit ipr^d J" ""j 

li H.ivr vnu *>ver bad nirm>il tratnfrii> In "Pm l^lun To n hi n£ H ' * Q ve« Q no 



t'ART 1 _ JnSTHIpCT lONAl PRPM^AH^ 

I nat ni( t ional Pr<*YtAm* ^ire t hc>* o pri^r<VK^ in wh ii b vour r>b - eel ^v*? ii to 
in< «it#a*f bfhavlorv; eitber hv teacMnP n.v b SnvlnJ and ^lt*ns» nr iiBpfi>ving 
ImtI iimunr c tn iltNlft -md h<*hnv1 or* wbin ft tht h ^todnnf hhin Previ otm a^quirr'd, 
Ni n r Jm luded r ir** prriffT im^ (U*ft1»tned to iln rr^t- or ^UdiinHite behavior- 

I T*iUer«?nt a*>ttingfi fnr thtii v Uufl U rt f d LommMh>n ,irn ^oi( rib*>d b*>lov 

l(M*3C e^t tnt^t e tht h ptir< ofit ,i^i^ nf v«>ot i M 1 1 I im rm t ion a I t ine rnndui t*>di 
tn mil h q*?tt1nft> rrti*>nLn^i> 



I tid iv Idui I respnnfles In i i h 1 sii-t t inp 
t f evn hor worlt in(f •* I ini4 ui t b hi *i mU'iii i 

individual r4»al>on*4»^ ,il na*t»TH*l "Ppi rtuni t 

( n*iiy be 1 l f small <>r 1 irp^ croo(> h in»t sfmhnt 

h-i* *ippnrtitnl ty mi n>HOH>tiif l t \M\ idn tit vi 

Individual r*?ftpon»i?rt In frou? tottinv^ 
iinniv be sm.ili or ] H irCii ^roupi* hut r n h 
^Indent h^a a titron 1 

rnltion repPrnAeA fn *i ^rr>tij» Pirtln^ 

Hm.iV he 4ivilJ Lir»i" Kutipfii lu»t t^^iillv 

^ t mifTi t ^ t «r<p(»nd t (tf ( t fn r ) 

nt hut H p J eniie dem i lht h ^ 



□ 
□ 

□ 
□ 



Z 
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PART ^' FVALUATFfW 



TMs section: concern* the way you evHua(e *our anient*' pmnreas 
neettnft his/her educational nbJectlYes Hnr iudinp; hut not United to, the 



In 



On *t>*t percentage of Ynur instructional pi^ftr.ir * do yopj t s your .URistrint 
or yovr smdent tearber usually rolled rtt leu*f <;ome (lata on attulent 
p«*rf n nuance 1 



B 



9^-10OT of the protfr-una 



□ » 



»5-<*4t 



Z] s °- 7ii D th * n itn 



■Bit 



WT*en yw do collect cuflu* data on an I nut rur t Ififlit I fti^Mi, annul h w nftfn 
f* it collert«r 

| | everY tlm the inst rurt ional ProfUasi J* 
ihot)t eYerY tkf the program in run 
|J about half the t i*Hf the Proftra» ia run 
| | ore JflJ i-nj 1 Iv 

lor Initial »M^^fnt * dJ^Kncstlr puipo«it«« and/nr Font-t^t nnlv 
J^j ot her fple.Mi* di**»rihe) 



When you collect data on lfiattiHtlOrt.il p*rf (.rpi.Ptu do yog usually 
coJierr data: 



Check_ out 

on every trial 
Q on the ffrat trrt) -nl* 
f"] on the lapt trial on 1 v 
Q oth*r <Plea*e d*4irlhi'} 



Q^J on a r/indivm number of trials 
on flpe<1a1 "pr^hV tH*l» f >nlv 
All r r f ,p J 4 dppr I nl 1, a ate annUnt t*f t Irk* 
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2 Afipr<ixlin<il#)Y how often ta faCH Inat rtirt inna I progta* typically run' 

fht'flc one 

£2J whenever a "natural occasion'* for the tarftrt behavior occurs 
[ two or nor* Preset sessions per dav 
J^j onp Preset session per day 

four preset *o**iiun* per w*eV 
[ thre<» pre**: nepslims p*r *trrk 
^ ] two preset pensions per v/erk 
[ on* Preset season per we**lt 

other (please d>*rrtbe) 



1 Approximately how manv nppor tuni t Iti <,r "rhanrea** to perrom nr respond 
with the tarKet heh.iYinr does the student Osu.'l Iy have In anv one 
Instructional perinrt* 

rhr* *■ onp 

[ '"[ J-^ trUlx [J S-Q trials [_J 10-1\ trials Q ]n-?0 trials 

mere than 20 trlala 
[J mtinv as pP^thle within a tiven period of titie* but usually 

>t le.mt | irLila 
j I drprnda nn wh.it fprfppens fn the '*n.ilurj I'* m raslnnr but uauatl> 

I — I 

.ibout [ I tri.ila 

I 1 | 1 

J drpfnds on thr studentS Perfot ance, but uaunily about I I trials 

£2J otlu-r (pletsr <l« s< tih*') 



^ Appto* 1i *ii rlv ht»v n^nv hours P<t wfek do vpu sr end planning 0 r fhanJIlnj 
y.ti.i tTPsMitCtlnii.il t »r<iff c rtsm' |^ j tiiium per w^elt 
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4 fti'l<*f M 4 M^t ff different ri,it*i Ivpi'S I" l^^iw^ put i onr <U by tV?r 
t h it vmti noct 1 r* <jt,m( lv ^llvit, .1 1 v<> ( bv I In- t vpr v^u < <*1 J »n"t **rnnd 
"i, t I n'cjijrnt 1 v , * u1 <i' fin l^ivr (hi' him bli"b H vi«j nr\^r <c>liprt th,lt 

( ^< Lit ^nlt * 

|J ] i cunt of bph-ivinr respnn«t* (bt*w tiuinv Mm** tin* brh^vlnt h,*pprn** but m 

f < otjnt nf * rUH Mm 11,1 1 Iv for "c M il* in < HtniTLi*n' 4 > 

J J ltrv**N of Assist an< v I for i-KJmpli-, iiun.tlii? ch.i< numb it <i] prompts 

J J r.ttt or frpijurmv djn fb<*:i*vi-T (."unl* p*-r uri J t of tiff) 

f^"j t Ji*t> d.it a M^hH fin * iT tbir,H 1 



uhrn tv mr^v' thr Mtinnint fht- 
ir> F rr»j 1 wit* HF t 1 

uf<t*n ch^ fttodtfni h^^ rh 1 .Mm 1 

vh( n 1 tn* jnudfnt: cftnolNI b* mnvj'il 

pr»>Tr*i"JMt* nil H J ^ 



u □ u 

LI U □ 



whin ti> rtiftnicr chr l^r mr Ht^iil 

t^rffu W k< <'p t hr t (J [ ] [7] 



□ □ u 
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INSTRUCTIONAL HICRRRCHKS RCSCRflCH PROJCCT 



Unfvrraltr <*f Us*hinKtcn 



Wr^rl* JJflrlnUr Principal Iflv^t 11 .ilnr 



PR€S€NTflTION R€SPONS€ QU€STIONNfilR€ 



part 1 , cuutrn or xhpoi^at^ presjehtgp 



FART 2: ATTlTVDt TOWARD TKE PROCEDURES 

1* What i* your fteneral attitude towarda the aiperiwntal Procedures following 
the present at ionf 



Very 
Favor ab le 



Very 
Unfavorable 



These question* concrrn the tts$tvn(«) rhat you attended covering the 
ejtperlnentsl Procedure* Circle <me for each statement. 



2, How ^eful do you think the procedure ■iaht generally be to teacher*! 



Very Clear 



1, " ic? treat ion Presented on the 

r,eral purPose of the procedures wis- 

2, The information on C£llc£tlf»I " d 
chart_tf>j data waa; 

3, TV in format too on drawing Mnea- 
of ^pr oj^r« a was' 

*, The information on uslnR deci sion 
rulea was: 

5, The information on tatabl l e n lng 
instruct iOfl*! forma t* waft? 

6, Tha information on the diffe rent type* 
of Instructional procedures ws«; 

7,, The Information on I mpleme nting 
instructions! *trs '^le» wss: 

3, rnformatfon on uairti the h ajtdboc L ; 

PMi-t^AV* *** information presented w*« 

10, General Coaments? 



Confusing 



Very Useful 
5 



Hot at all 
Useful 



3 How applicable do you think etch of t ha following would be to YOUR sKustlonT 



a, Overall* the a*perl«ental 
procedures would be; 

b, ttealgnlnt initial inatruct lorta] 
formats according to the 
information presented; 

c, Conducting Inat ructions! program* 
with individual student responses 
on a re&unr basis; 

d, Collecting performance data 
during an instructional program; 

e, Charting parformanca data *e 
described on a Tabular baala; 

f* Uelnft daclalon rules baaed on 
llne*-of -progress 

g , Changing marruc t'onsl nethodc 
as de»c rlbed: 



Very 
Applicable 



#ot at all 
Applicable 



0 
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*. tfcnild )fou Hlfp additional information on ant Part of the Procedure*' 

C 110 C E?!^> M«m cn#c* tfcoa* Ch«C apply*; and * *#fhber of th# 
reneatch ataff v!12 contact you sharfJv, 

j ufting tba handbook 

J daalejnin* initial Jnat ruct ions 1 formats 

Q aalactin* * dat* typ* 

J^j iaple»*nting inar ruct ional Prot*dmCfl 

j col Zact Jflft par fortune* r*ata 

j char tinft Parfoimanc* dat* 

J^j drawing Performance pattern* 

£J uainft daciafon rul*« 

□Applying th* B*n»ral china** to aPtrific 
proAraaa 

Q oth#r: (daacrib briefly) 



Thank you I 
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INSTRUCTIONAL HI€AAftCHI€S ft€S€ftftCH PAOJCCT 

University of V«hlnKton Horria HarinKV Principal Invaatiftator 

HANDBOOK QU€STIONNftlfl€ 



Approximately hov much r 1h hav* Joti aPant taviwina, the information i n tha Bamboo*? 
ainurea four* othrr 

Tbaae quear loo* concetn th« clarity of informarion on the k*Perim*ntal Procaturaa 
aa presented Jn the Handbook, Pletae circle on* for each ttitfienr. 

Very Claar Confoaina, 

1, The information chi Hov To Hefce An 

In*ttuctional Detiaion (p<5-?> it: 5*321 

2h Th# ftulea for Hate and the Quit* lleference 

Guide for Hul*« for flare are: 5*121 

? The flu It » for Petcarit Correct ipd tha 
Quick Reference Guide for Aulea tor 

percent Correct erti 5*321 

4, The Quick reference Guidea fot th*o|» 

Sttategie* <p.i0-U) arei 5*311 

5, The informarion on drawinl linea-of -proftreaa 
(p. 1 5-14 > » and determining percent ca-rrett 
(p, 25) * performance variability (p,2W T > 

and aharp deceleration in correcra <p<2$) ia: 5*321 

6, Tb* inforaation on Gaibaral Conaiderat foot for 

Instructional Formata (p-2»-35) H: 5*321 

The information ort Acquiairion (p.]6-6]> ia: 5*521 
B, Th* information on Fluency-Bui Iding <p,64-95> ia: 5 * 3 2 1 

9, Th* information on Compliance {p,*7-n5> ia: 5 * J 2 I 

Vr would T jftt your t^amejita and auftgeatf^n* regarding th* ttartdboo**- Flaw* ua* cb* 
back of thl n ahi*t 
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INSTRUCTIONAL HKRAACHKS ft€S€flRCH PAOJCCT 

l^nl* emit* of WaihinKti.n Muffle H.ir Infw frim it»at liwrst lj" unr 

TRIfU P6RIOD QU€STIONNfllR€ 

Tnd^y'a dai r ^ 

h ''Trial'* ol the pror-edurea atftht involve nriV r;inF.e of .irtlvit ?a * f r - ipplvlnft 
the procedure* wirh one itudcnt Jut one program f*ir * frw uppVwr to <i 1 1 ^udenti 
And atl pTt>rr-i»A for the n ** of [he School vfar. Ymi uV* fd<* nn your level < T f 
Jhvnlvement 

1 Tkj you rinn on us I of the Proce^urea for * trial period* 
j^J (JW^> on 10 lh e quearlone below 

C55Z^> &° <*>n t0 ? 



2* Asslsrance 1* AvAlUMr to r rhone participants who wlnh TO tr> the procedure* 
Asststance 1* available if t«u need help In npplvtnft the Procedure* fn vtmr 
situation. Are you Interested in rereivln* anv foll*-up easiatame* 

U L^L f ^> * ■enber of the research staff wtxl contort You atifcrtl* 

1 At thli rt»e, f what tVpe of trial per**>d do *ou pla*." v <iu can tr nf rimrip, 
iltfi your pl^nfi at an* lime* Choose a, : b or c 

a Q t5XX£> [_J atudent(a) and L_j pronr.iM «) for ^l-Ul 

BejttfinloR 0 fi 



day nonrh ve^r 

b + J^j |^i|o>£iit3^> A ■ewber of the revearrh staff wl II foot an yr.u short 

c (qthe ^ A member of the research ataH will contact vou shortly 

* Are other teacher* tn >our achool u*tln£ these procedure** 
Q ve* £""J no Q don't fcnow 

S t>r» Vou feel that the adalnlM ration unuld provide aufpirt for the use of 
these Procedure* 



Thank your r , 
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iiJT c i>nr 1 OoH fliippHTr t jjnd a*q 1 -n C an< 1 



Mil uut this p3p<* otilr 1! r<iu do WOT Plan to une the procedures* 

; An, i<lWr tuiihi'i* in vmir nbnf.l using three procedures' 
[ ] v ' s [ "I on [J don't Vnow 

1 vim that the ridtnlnist rai Inn In Jour wrhool inr district would 

prnvltt* support f^r "ht" n*»e of ibime pr i>red<nTea l> 



4 Umf you do ROT plan on geln^ the proccdurea* we would be tntereated In 
Vhiit rea«oo«> Pleaae check *U of the hone* below which aPPly> if anVr 
ANli/ftR provide a written reason J " ih* space Provided. 

□t feel that thf oroceduree will take too M ?h risw, which I 
lust don't have 

I 1 fed that thi procedures are not appl icnbie t« wry studfnte^ 
J 1 Irel that the procedures *re nor c<*Patihle with *t curricu]u«> 

nl feel that the procedure* are nor compatible with *t current 
in*tt Tuct lottAi t*TAct Ices ♦ 

f~ I 1 do not f**l ih* nwd to use the procedure« + 

□ 1 do not thlnt that the procedures will IsiProvc tbe performance 
av students 

Q 1 do not gnderatflid how ro applv rhe procedures tn »>■ sltuat ion ♦ 
In the apact below,- please deirrtbe either renSotis you vrV have for not ualnfL 



Th.ink V.m l->r Vrtur p it t 1 1 I In I I an i- thi* prnlerl. 
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INSTRUCTIONAL HIERARCHIES RESEARCH PROJECT 

Dnlv*r*t*v rf WKhlnPlcfi N<*rrM ttnrlnK, Principal Invtct lr<*tor 



PROC€DUR€S QUESTIONNAIRE 



PAJH I'* ED UC ATIONAL SETTING 



1% For how Mnv atudtnta in th* following csttgottts do Jfou provide DIRECT 
•*rvic* (instruction or thtraPv) 7 



Ho* "anv + 



|^ j "learning disabled'* 1 ^] profoundly handicapped 

|^ j *l Idly handicapped |^ j 



other 



|~~^~~j#od*ret*ly nat*dtcaPp*d 



□ 



average tPtaj «tud*r>r* 
»*v*r*lJf handicapped ptr ytar 



2* Chronological ag**- Froa 

youngest 

3 Addtcional atajf uiuall* working yich rou- 



O 

| | ytara to p j Y*t\fm 
ounfl^st ol dest 



avtragt on any LZ£LSjL^ 
scho ol jay 



Ntfobtr of P*io* aaaiatanta: j for a total of | j 

j for « tntal of | houta 



Ntmbtr of volunt«ra; 



Kuabtr of oth*t»' foe 



hours P*t 

p*r vrrk 



t a total of noura per v**k 

at atrvit* pe f ionnM physical 
munication dlaotdcra lp*rl*l ici q 
hetapy) to on* o( »or* o( your s 

j""! ftp |~j p?Ff"^ > for an average total of hours 



ft*. If you ar* a claaaroo» t*ach*r t Jo" ip'ci 

thttaplsta* occupational » h*rjipiata f/ comuntcat ton dlsotdtra Ip*r1*l ici 
p.tovic> dir*ct_«fvl£JE < instruct ion or th*tapy) to en* ol »or* of. your atvlpnin* 



• 



5* Do ynu t*ne as a eWrvUor foT unl vtrsJc v/col l*g*/high achooi atud*nt 
teachers or pMcticurn student*' 

r~~j No r~~j rTTT^b Avtr^* nunfttr p*r quarter |^ | tor j ~j total houra 

»ART U» 05E O F THF KASpBOQy 

Th* foUoving qu^tiona concern in* utt/ulhtss of th* Handbook during thr Trial 
P**iod + ri^.ist circl* onf for *acn st*t*w*nt + 



The information nTtatnttd on* 



Very 
Useful 



1* How to Hak* ar + Instructional 

Decision is S 

2 + Quick Rtftrtnt* Guides fOT change 

Strate(ti*$ <p,lG-lft> arc: 5 

J Drawing Un*«t of Progress 

{o.lWM U*- 5 

* * D*t*rainlng percent Correct 

*;> + :M la 5 

5. D*t*rraining Perforaiance 

Variability (p + 26-27> is 5 

6 fiharP Deceleration In 

CorTPCH iP.lft) 18: 5 

7< Central Cons id* rat tona for 

Inatturtirnal Forowts (P . 29-35) la: 5 

P Acquisition (p 36-63) i* + * 

^ r Flucnry-Boildinft <p-6ft-*M i* + 5 



10 Cotr T ,lljnri> ( p .9 7- 1 1 5) Ik 



Not 



Ua< 



vould aPrtrciate »ny coutt-nts or eug8c«t iona you nay have regarding the oec 

of the H.Whndk during tht Tttal Period* 
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FART 111- ATTlTtDE TOWARDS THf PBOCEDirRES 



Now that you have had * chance to trv the procedures to vour classroom, what 
1* your general attitude toward 



VerV 
Favorable 



1. tJting the procedure* when 
instructional program* are S 
tatablished* 

2, Collecting direc: performance 
data' 5 

3* Charting performance data 7 5 

Urawln; Hln1»u» "Celeration 

(Aid > tinea' S 

5. Drawing Llnea of PrcFreis in 

order to aPPl/ rules'* 5 

6* Uatng the experimental decision 

rul-s* 5 

7, Uaing the chant* tt ratepl ea for 

instructional programs 1 S 

B. Overall* 5 



Unfavorable 
2 1 



No 
Opinion 



jftftT J.V US I K G THE PROCEDURES 

1. How *in/ weeks have you uted (did you use) the experimental procedure** 
| ~ j veeks 

2* How san/ of /our atudenta were Involved 



• □ 

3, What vat the total nunfctr of program* Involved' j^^j 
What rulta did you use 1 



atwdent* 

prograa 



JltuJea for ftate Data <On /ellow paper?. 
Bow often did you sake decisions? Ceiwrall/ ever/ 



| | data dayCsK 

Have yow com across data Pattern* whtch are not covered by the rules* [jlj 



uJe? for Percent Correct Units yellow PaPer) 



y^c ii4 



Mow often did you m*kt tfrrislors* Generally ever? 1 J data *aya. 

Have you roan? across data patterns whirh »re not r^.erad hy trie rulea* [_yJ[nJ 



FxPer latent al Rules for Use wttK rttniwr 'Celeration, 



.^alnately how «uch tia* havft you ap«it uklng Instructional de<:Ulon* 
and chanpinfc inttruc t Jonal Programs* 



hours Per weak 



How aots this co*pwre with the **oun£ of *1« you spent before wtfni the 
Proc enures' 



□ 



sore tlste 



leaf tin* 



□ 



the taste amount of tit* 



Indicate rr*. t^ral number of Prograais on which yow collected rhe following 
tvPf** of data: 



regular rate 
adjusted rate 



□ - 

| ""J percent correct 



□ 
□ 



duration (trial Mthod) 
duration (teasfon sethod) 



7, Overalls In youi* Opinion^ what impact have tTTe decision rulta had on pwPil 
progress as compared with their progress prior to the trial period* 



accelerated puPil orogreas 
decelerated PuPil Progress 
no di rf erence 

PART V; the FlTUfcE 

»e would like to w«e your data In order to colltct additional Information con- 
cerning the ef f ectlvenesa ol the Procedures, 

1 would you be willing *o tend us eoPles of /*ur pupil Perforsanct data and/or 
dec It ion record aheetti, 1 f we par for the caPylnK* 

/as (a timber of the research tea* will contaet J*u) 

2 Dp vnv plan to continue usinj; the exFerlvanral procedurea as a regular Part 

ff yu\n clasironm at i Ivl t tei* 
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3. Uoul<* voti Ilk* f o **t*nd th^ trial p*rioJ* 



ye* {a nrfhkvi of Xhr rfttjrch F tain vlll <rvnt^rt vou) 



□ 



if you do not plan on r*trn(*tnp thr tHa) Prriodr ind/or vcu do not plan on 
twins ihr procrduf« »ha Mr* J < lp*i ion 1" th'i fit udv ^ vf vould apprffintr 
knowing vour r^-i^n* 



Thank you for taklnfl thr time rompi*t* thl ^ qu. - 1 i^nn* i rf and for poor 
p«rt H i I fiat Ion in t hi i , iv 
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FIELD INITIATED RESEARCH STUDIES 



AN "INVESTIGATION OF STAGES OF LEARNING AND FACILITATING INSTRUCTIONAL EVENTS 
FOR THE SEVERELY/PROFOUNDLY HANDICAPPED 



Norris G* flaring, Principal Investigator 
Kathleen A, liberty, Project Coordinator 
Owen R, White, Research Associate 



DATA-BASED DECISION RULES FOR INSTRUCTIONAL 
PROGRAMS 

JCNE 1980 



The material contained herein and presented at the conference is based on 
research which is continuing* Results subject to reinterpretation at the 
drop of a data point. If you would like to be kept up to date on changes, 
please write c/o Experimental Education Unit, WJ-10, Child Development and 
Mental Retardation Center, University of Washington, Seattle, Washington 98193. 

ihe material contained herein was developed pursuant to a grant from the 
Bureau of Education for the Handicapped, U.S, Office of Education, Department 
of Health, Education and Welfare. Project No. 443CH6039A, Grant. Ho. G007500593. 
Contractors undertaking such project under government sponsorship are encouraged 
to express freely their professional judgement in the conduct of the project* 
Points of view or opinions stated do not, therefore, necessarily represent 
official positions of the Bureau of Education for the Handicapped* 
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PREREQUISITES FOR USE OF EXPERIMENTAL DECISION WLES 

/ 

1. Design programs to teach new behavlore or to improve performance. The rules ^ 
are not deelgned to be used with programs aimed at decelerating or eliminating ^ 
behavior. 

2. Flan for a consistent Instructional format which you will use each day. The 
plan should Include: the antecedent event* ( cues, atlmull, materials, *znt- 
ruction) you will use to get the behavior a tar ted, a precise definition of 
the behavior you want to aee, and the aubsequtnt event a (like praise and 
prompting) you will use to provide faeback and coneequatt performance). 

3. Provide the opportunity for the atudent to respond Individually and 
independently at least ten times during the Instructional aesalcm. You 
nay choose to provide the atudent with a very ahort period of time In 
which to respond, and then aaalst the student to raapond If he/ahe doesn't 
during the period you allow. Or, you may dealgn your program so that the 
student has the opportunity to perform juat a small part of the behavior 
Independently. If the atudent can not perform any part of the response 
Independently, you should consider moving to a different related aklil or 
to an easier eklll level. 

4. Conduct the Instructional aesslon at least once per day for aeveral daya 
without changing the format you aet up. The greater the number of opportunities, 
the faster the learning. The decision rules will tell you when you should 
change your Instructional plan. It la Important not to change on svary 
trial or every day (unless the atudent meeta criteria) to give the plan 4 
chance to work* Changing tactlca every trial and/or every day may confuse 
the student. (You may want to change every trial If you are programming for 
generalization, but then the changing Itself would be consistent and part 
of your plan.) 

5* Determine the performance aim — wh:'t type of performance Is deslreable? ficv 
accurately should the atudent perform? How fluently? Usually accuracy 
criterion by themselves (e.g. 80% correct) are not sufficient to ensure 
mastery and maintenance. Some time component will ahow fluency, either 
use rate, duration or latency In determining performance alms. Use 
the performance of nonhandicapped peers as guidelines In determining aims; 
using lower alms will ensure "retarded" performance. 

6. Select an appropriate target date, by which you want the atudent to achieve 
the performance aim. At flrat, It may be difficult to aelect a date* but 
with practice, you will be able to select dates that are both reasonable, 
achievable and deslreable* 

7. Collect and chart performance data. Try to collect data ^3ch time you 
conduct the program. Chart the data so that you will be able to visually 
determine the progress of the student to the goal. The decision rules are 
designed to be used with ratio Interval chart paper. 

l..'u 
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Data types 



RATE DATA 

When to collect it 

Rate data should be collected when rate or frequency of performance is the most 
important aspect of fluency, and when the situation is structured so that the student 
is "free" to perform during the entire observation period. It should not be used 
when the instructional or intervention plan specifies antecedents or consequences 
which interrupt performance. For example, rate would be collected in a sorting pro- 
gram when the behaver was given all of the materials necessary to sort at the be- 
ginning of the observation period. If, however, the manager were required for some 
reason to give the student the materials one at a time, or to cotisequate each "sort" 
with prompts and/or food, the antecedents and consequences Interrupt the student's 
performance, and thus adjusted rate data should be used instead. 

How to collect it 

? 

a. Determine the length of time Jou will provide the behaver in which to 
respond. Usually the same length of time is provided each session (e.g. a ten 
minute period). Or, you may decide to time how long it takes the behaver to complete 
a certain number of responses (length of time for ten trials, for example). 

b. Arrange the setting so that the behaver is free to respond during the entire 
period. Usually this will entail providing enough "work" so that the behaver will 
not finish before the time is up. 

c. Give the signal to start performing, and atart the stopwatch. Do not wait 
for the student to start, but start the watch at once. 

d. Count the number of responses the student makes, both correctly and incorrectly, 
either during the timing itself r or by checking the work after the timing is completed. 

e. Stop the stopwatch at the end of the predetermined observation period, and 
give the signal to stop performing to the behaver* Record the length of time and the 
frequency counts. 

How to chart it 

1. If thp length of time you recorded involves "seconds", either when it is less 
than one minute (e.g. 30 seconds), or a whole minute plus a fraction of a minute (e.g. 
5 minutes and 40 seaonds), use the attached tablft to convert the seconds to a decimal 
fraction (e.g. 30 seconds * *5; 5 minutes 40 seconds - 5.68). Use the decimal fraction 
in performing all of the calculations described. 

2. Fill in the information on name of behaver, manager, movement cycle, date, 
and so on. Use a separate chart for each movement cycle if possible, to avoid con- 
fusion when using decision rules. (Do not use different colored pencils for esch re- 
sponse, for example). 

3* Record the length of observation time on the chart by: 



length of time 

If you observed for longer than one minute, thr result of this division will be a 
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decimal fraction (1/5.66 * .18)* If you observed for less than one minute, the 
result of this division will be a whole number {l/,5 * 2). Find the result on the 
left-hand vertical scale on the chart. Find the day line for the date of your data. 
Record the length of observation time as a horizontal bar across the day line. 

cording the length of observation time is very important since it serves as one 
of the bases for comparing changes in performance. 

4. Record the rate of performance by: count the num ber of correct response, 

length of time 

For example, if you recorded 10 correct responses in 5,68 minutes, the rate of 
correct responses would be 10/5.68 * 1,7, If you recorded 7 correct responses in 30 
seconds, the rate of correct responses would be 7/, 5 * K. Find the day line for the 
date of the data. Record the frequency of correct responses as a dot . The frequency 
will be above the bar denoting the observation time, The rate of incorrect responses 
is calculated in the same way, and recorded on the chart as an x. 

If you recorded only 1 response, the dot or x will fall directly on the 
horizontal bar which denotes length of observation. If you recorded 0 responses, 
chart a ? directly below the horizontal bar. The ? is used since do not know, 
and cannot calculate, the rate of responding. 



ADJUSTED RATE DATA Trial Method 
When to collect it 

This type of data should be collected when performance of the behavior is inter- 
rupted by antecedents and consequences, and when rate is the critical time dimension. 
For example, during the beginning stages of a sorting program, each object sorted may 
be consequated with a squirt of juice in the pupil's mouth* In this case the delivery 
of the consequence interrupts the student's performance period, and adjusted rate data 
should be collected. Or, if each object to be sorted is given separately to the stu- 
dent, the student must wait for the object before sorting, the* delivery of the ante- 
cedents interrupts performance in this case. 

How to collect it 

a* Arrange the materials for one trial (or two or three, depending on plan), 
provide antecedents and give signal to respond. 

b. Start the stopwatch immediately following the signal to respond. 

c. Stop the stopwatch when the pupil completes the response (either correctly 
or incorrectly) or when the allowable latency period has expired, 

d. Record whether or not the trial was correct or error, consequate according to 
plan, and go on to the iiext trial. It is not necessary to record the time for each 
trial; time can be accumulated on the watch and recorded at the end of the session. 

How to chart it 

Chart it just as you would chart "regular rate 1 * data. 



LATENCY DATA 
How to collect it 

a. Latency data measures the time between the end of the timulus or direction 

1 9 * 



to perform the behavior and the behavior itself. Teacher says "Hi" latency "Hi" 
says the student * 

b. Start watch at end of stimulus, and stop watch at beginning of behavior* 

c* For general usage: Do not reset watch at end of each trial; accumulate time 

on watch. Keep track of th* number of trials. Record 

time at end of all trials* 

Hov to chart it 

Chart average latency per trial OR 

average latency per correct response and average latency per ertor response* 
These are equivalent to the average frequency charted for rste data* 

1* Relabel the vertical sxis "Average Latency in Seconds", and fill in the 
identification information* 

2* Determine performance ceiling* 

In most latency programs there is an allowable latency limit established as 
part of the program* Usually this limit is S seconds per trial* If no response has 
been initiated S seconds after the direction, the contingency for no response (e*g* a 
mandate) is implemented* The latency limit must be charted as a ceiling, since the 
contingency will prevent responding when the limit expires 

a* Find the limit on the left hand scale of th* chart* If the limit is less 

than 1 second , it will be below the 1 line* 
b* Mark the limit as a dashed line across the chart* 
3* Convert the total latency time into the appropriate decimal equivalents, 
utilizing the table attached (e.g* 12 seconds » *2}* 

4, Tind the average latency per trial: * " 

total latency 
number of tr ial s 

Find the result of the calculations on the left-hand vertical scale of the 
chart* Find the appropriate day line* Chart the result with a dot if the prcgram is 
designed to accelerate latency, and as an x if the program is designed to decelerate 
latency* If 0 latency is recorded, chart a ? above the ceiling* 



DURATION DATA (Session method) 
UTien to collect it 

Collect duration (session) data when the length of time the behavior lasts is 
the most important aspect of fluency, and when the situation is structured so that 
the student is "free" to perform during the entire session* For example, length of 
time with head held in midline would be appropriate for the duration session method 
if the observer watched the student and did not attempt to hold the student's head 
in place, and if the student were sble to move his head up once it had fallen* If 
the student were unable to move his head up, then the trial method would be most 
appropriate* Duration data, session: this method of collecting duration data is 
usually appropriate when the aim of the program is to decelerate the duration of 
various self-help skills* The aim of a shoe-tying program mry be, for example, to 
decelerate time spent tying a shoe from three minutes to five seconds* 

How to collect it 

Start watch »>hen the behaver begins performing the desired behavior and stop 
watch when the behavior stops. Accumulate time on the watch. ALSO, record total 
•ess ion length. Record time of day when you begin fqlUctiig dats and time of day 
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vhen you stop collecting data. In the example, session time vould include both time 
on task and time off task, 

Hov to chart it 

1. Relabel the vertical axis of the chart: "Minutes Duration Per Session". 
Fill in the identification information. 

2. Find the "session time". The length of the session is a "performance 
ceiling", since the student cannot record more time than the length of the session* 
Find the length of the session time, in minutes, in the left-hand scale of the chaH. 
Chart the ceiling as a bar across the day line. If the session length is the sane 
every day, chart a dashed line across the chart. 

3. Find the "duration'*. This is the length of time in minutes and decimal 
equivalents of minutes that the behavior occurred during the session. Find the duration 

in the left hand scale of the chart. Plot it at that point on the appropriate day, using 
a dot if the aim is to accelerate duration and an x if the aim is to declerate duration. 
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DURATION DATA Trial Method 
When to collect it 



This type of data vould be used vhen performance of the behavior is interrupted 
by antecedents and consequences. For example, in a "Hold head in midline" program 
designed to increase duration, an atonic neck reflex vhich pulled the head out of 
midline vould both end the trial and result in the need for the manager to normalize 
the pupil f s muscle tone prior to the next trial. In another example designed to de- 
crease duration, the pupil's completion of zipping his jacket ended the "coat on" 
trial. The jacket had to be removed before the next trial could begin. In such cases, 



use the trial method 
Hov to collect it 

Start vatch at beginning of trial/behavior and stop vatch at end of trial/behavior. 
Accumulate time on vatch. If separate duration for correct and error trials is desired, 
each trial must be recorded separately. 

Hov to chart it 

1. Relabel the chart "Average Duration in Minutes", and fill in the identification 
information. 

2. Convert any seconds to decimal equivalents, using the attached table, 

3. Find the average duration per trial by: 

total duration 
number of trials 

Find the result on the left-hand scale of the char*, and find the appropriate day line. 
Plot the data as a dot if the aim is to accelerate duration ahd as an x if the aim is 
to decelerate duration. 

4* If you record a total of 0 seconds duration, you vill need to determine the 
smallest amount of time your stopvatch can accurately record. Although the accuracy 
may vary, you may assume that tht? smallest amount of time is 1/10 of a second. To 
find vhere 1/10 of a second is, convert it to its decimal equivalent (1/10 divided 
by .0166/10 * .0016), find that fraction on the left-hand scale of the chart, and chart 
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it as a horizontal bar across the day line- Record "0 duration* 1 as a ? just below 
that lim . 

5. If you desire separate plots for "correct" and "error" trials , follow the 
directions for step 3 separately for "number of correct trials" and for "number of 
error trials". 
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USING DATA DECISION RULES TO ACCELERATE PERFORMANCE RATE 



U Determine the behavior to be changed, the performance aim, and the 

instructional format. Select a target date. Document the procedures, 

2. On your chart, find the vertical day line which is the calendar date you 
selected for the target date. Find the horizontal line equivalent to your 
performance aim. Draw a -A- at the intersection of the aim and the date. 

3* Implement the program, collect and chart three days of data, 

V To set the minimum 'ceteration line, follow these steps: 

a. Find the second data day. 

b Find the second highest correct performance, If you are setting the aim 
for corrects. If you are setting the aim for errors, find the second 
highest err" ,rformance. 

c. From the intersection of the second data day and the second highest 
performance, draw a line to the -A- 

This line describes the minimum 'celeratlon: the change aim. 

5* Continue running the program and collecting and charting the data. 

Since the minimum 'celeratlon line describes the desired change not the 
actual change, each day that you chart you should decide. Keep the following 
in mind when you are charting: 



a. Is the pupil at aim ? If the pupil reaches the aim, you should move to 
the next curricutar step or program as soon as possible. Delaying may 
set the stage for poor performance. The chart should tell you before* 
hand that the student is getting close to the aim. Return to Step 1 for next 

program step. 

if the pupil is not at aim, go on to the following questions. 

b. Is the pupil progressing? A pupil may be above the minimum 'celeratlon 
line but not progressing. If you think the pupil may not be progressing, 
draw a 1 ine-of -progress for the most recent six days of data. If that 
line is Hat or going In the opposite direction from the one you desire, 
make a change using the procedures described below. This step Is very 
important, especially if the student's actual progress is not very "near" 
to the minimum 'celeratlon line. 

c. Has the pupil's performance fallen below the minimum 'celeration line 
(above for deceleration projects) for t hree consecutive data days? 

If the answer to this question is M yes M , !t is probably time to make 
a change in your program. 

6. If you decide to make a change in the instructional format, yoj can use 
the decision rules to help decide what area of the format Is most likely 
to need changes . 

a. Draw a 1 ine-of-progress for the six most recent data days. This may 
include the three below the Une and # the three before those three. 

b. Use the attached flow chart and list of format considerations for ideas 
on what to change. 

12o 
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7* Once you have selected new procedures, document them. Now set a "new 11 
minimum 'celeration line. Follow these steps: 

a. Implement the new pro^adures and collect the performance data, 

b. On your chart draw a vertical line 1/2 day before the first day of the 
new procedures * 

c. Find the second highest day of the last three days of data (the last 
three in the "old" procedures). 

d. Hake a mark on the vertical line which ts "equal to" the performance of 
the second highest day of the last three days of data. 

e. Now draw a line from the mark to the IT . This will make the new 
1 ine steeper. 

If you do not want a steeper line, draw the new line parallel to the 
old 1 ine. This wi 1 1 change your target date. Draw a -/L at the inter* 
section of the new line, the performance aim and 
the new date * 

This procedure prevents the new procedures from having a terrible 
effect for more than three days. If the new procedures are not 
effective, the behaver will be below the new line for three consecutive 
days. If this happens, make a change of the same type as you made 
before. For example, if you made a consequence change, make another 
consequence change. Follow procedures ?a~7e again. If the behaver 
still ends up below, the rule Is not working at all, and use your 
Intul t1on~—we realty don't know how to advise you, since this hasn't 
happened to us. Please call us tf it happens to you! 

6* Continue conducting the program and collecting and charting performance 
data. Remember to ask yourself questions about the performance as your 
are charting* AND REMEMBER, THERE ARE NO MAGIC RULES. ..you may have more 
success by changing according to your intuit1tion r knowledge and experience. 
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Minimum 'Celeration Line 



from: White, O.R. £ Haring, N<G. 

Except 1 pna 1 Teach i ng , Charles 
E. Merritl, J976 




• t Se Uheaim star ,, thr intersection 
c <nm Mtr Anfi dim o\*le 



8. Draw a new aim star , if wees- 
sarv, ai the point where the new 
minimum celeration line 
intersects the dim rate. 



9. If a change in the program is 
madelor any other reason than 
a failure to met*' the minimum 
^celeration criterion , draw 1 
phase -change line as befo** but 
continue the otd minimum 
"celeration hne without change. 



Whenever three successive 
rates fall below the minimum 
celeration line , change the 
instructional plan 



Draw a phase * 
change jmc -7 T /? days 
before thr hrsl 
rated day under the 
new plan. 




Locate a new start 
mark by placing a 
cJoTon the inters 
section of the first 
rated day umfer the 
new plan and the 
mid -rate of the last 
trvee assessments 
under the old plan. 



Draw a new mtnjmum 'celeration line ,, 
either be:ween thr "Id aim star and the 
new start mark or parallel to the old 
minimum celeration Imp (the second 
option is illustrated here* 
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6 DAY LINE OF PROGRESS 
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J consecutive days 

beio<d minimum 
celeralion ///?£ 

3 days preceding 

fwtd'dacj of first 3 chtjs 
mid - level oP first 3 days 
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intersection 

mid -day of last 3 days 
mid- level of fast 3 days, 
ohei 2 are -tie same 
that m/I be l/?e w^k 

Intersection 
connect intersections 
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SHARP DECELERATION IN CORRECTS 

A ^harp deceleration In correct performance usually Involves going 
from a fairly high level of performance to a low level. Example' 
are shown below. 
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PERFORMANCE VARIABILITY 

Performance variability Is determined by comparing the data points with the 
1 1 ne-of-progress , If most of the data points are further away from the line 
than the distance between 2 and 3 on your chart, variability Is HIGH, 
Examples are shown below and on the next page, 

PERFORMANCE VARIABILITY REFERS ONLY TO "CORRECT 11 PERFORMANCE 




PERFORM A NCR VA R I ARIL IT Y 
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Behaver: Severely Retarded Hale 
Age 18 

Behavior: Picks up cued cup 

(2 choice discrimination) 
Aim: 60 correct per minute, 1 or 0 
errors 
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Behaver: Profound!' Handicapped Girt 
Age i 7 

Behavior: Picks op spoon, holds op 

for 2 seconds 
Aim: 20 correct per mtnote, 1 or 0 
errors (drops spoon) 
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SUCCESSIVE CALENDAR DAYS 
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PERCENT CORRECT 




The Percent Correct should he above 832 for the majority of data days in order 
to answer this question "YeB." 
To determine Percent Correct: 

Method A: For each day of data, add the total correct to the total 
errors for the total trials. Divide the total correct by 
tfte total trials. 

Method B: Use the "Z Meter" on top of your "EEU Rate Plotter". Put the 
"50" line on the errors and read the % Correct next to the 
"plot dot" for corrects. Of course, if errors are higher than 
corrects, corrects will be less than 50% and the plotter will 
not work* If corrects are at "zero" (below the floor), corrects 
are at 100? and the plotter is unnecessary* 
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INSTRUCTIONAL H!€RRRCHi€S R€S€RRCH PROJECT 



Universit** of WashinRton 



Norrls Hartng, Principal Investigator 
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EXPERIMENTAL RULES FOR USE WITH MINIMUM 

ACCELERATE RATE OF RESPONDING 



CELERATION 
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DESIGNING AND CHANGING INSTRUCTIONAL FORMATS 



General Considerations 

I* Are you provMing the opportunity for independent responding? Have you 

established an allowable latency period? 
2* Do you have appropriate consequences? Do you have different consequences 

for correct and incorrect responses? 
3. Are you using appropriate signals to get the behavior started? 
4* Are your materials natural and age-appropriate? 
5* Is the setting for instruction appropriate? 

Format Considerations for Acquisition 

1* Reinforce accurate performance, 

2. Provide sufficient response opportunities. 

3* Provide as little assistance as possible 

4* Provide as little extra information as required 

5* Consider the entire behavior 

6* Consider generalization when you choose constant or varied stimulus events 

Strategies for Acquisition Problems 

Provide aoditional information as a 
consequence for error responses 
antecedent to the opportunity to respond 
or conjugate with responding 

1* Change verbal/signed/gestural direction or signal 

2. Add gestural cues 

3* Add verbal cues/stress key vords 

4* Add a permanent model 

5* Add a manager demonstration 

6. Add physical prompts 

7. Add an assisted demonstration 

8* Add position/color/emphasis cues to instructional materials 

Format. Considerations for Fluency-Building 

K Reinforce fluent performance* 

2, Increase opportunities to perform the behavior (drill & practice) 

3* Increase the freedom to perform the behavior 

4» Increase the rate of teacher-presentation 

5. Consider generalization 

Strategies for Fluency-Building Problems 

1, Provide directions for fluency before/durirg performance 

2. Add/change consequences to motivate correct/fluent responses* Use a 
"reinforcer survey" to identify possible consequences: provide "variable* 1 
consequences* a different type each tirae> hide the prize until it is earned 
use conjugate consequation ♦ 
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5. 
6. 

7. 
8. 



Change the schedule for consequation of correct /fluent responses* Increase the 
ratio or interval of the schedule; use a variable schedule; delay conjugate 
consequeation. 

Implement a changing aim strategy, increasing the amount of behavior required 
tc receive consequation daily; tell or show the student how much "work' 1 she/ 
he must finish 

Xncritise th^ number of response opportunities* 

Increase practice, v drill 

Work with a more competent peer. 

Add/change consequences for disfluent/incorrect responses 



Strategies for Compliance Problems 



2, 

3, 

5. 
6. 



Hove tc a more difficult skill level. Are you sure this skill level is the 
correct one? Ars your criteria tcj high? Do you require a lot of days at 
criteria? Is the student bored? 

Change or add a motivating consequence for correct/fluent performance* 
Change the schedule for consequation of correct responses to avoid satiation* 
Institute a response cost procedure (Gain for ccrrect/f luent * lose for error/ 
disf luent). 

Eliminate competing consequences* 

Add a time limit for no responses /change or add a motivating consequence for 
no responses and for errors (Caution; parental/guardian permission adviaed prior 
to implementation of aversive, negatively reinforcing or punishing consequences). 
Institute "all day' 1 procedures for compliance. 

Avoid: Tuoving to an easier skill level; repeating the instructions; coaxing 

or prod .ting the response; providing assistance; completing the response yourself; 

threatening the student; yarning the student. 
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INSTRUCTIONAL HI€RRRCHI€S R€S€RRCH PROJCCT 

University i>f Washington Nnrris llaring, Principal InvestiRator 
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ACCELERATE DURATION OF RESPONDING 
EXPERIMENTAL RULES FOR USE WITH MINIMUM CELERATION 
144 145 
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SUPPLEMENTAL RULE SHEET: ACCELERATE DURATION OF RESPONDING 

The Supplemental Rule Sheet is an expanded text version of the accompanying 
flow sheet for "Accelerate Duration of Responding' 1 * Each diamond-shaped figure 
represents a decision point followed by a dhort explanation which should help 
you make an instructional decision* Since much of the information has been 
presented in the two inserts "Using Data Decision Rules" snd "Data Types", a 
quick review of these sheets would be helpful* 



AT Al/A? 



If the student hss met aim, you should move to the next 
curricular step or program as soon as possible* If the 
student has not met aim t go on to the next question* 




Is the student progressing? A student may be abo"e the 
minimum 'celeration line V.it not progressing. If you 
think the student may not be progressing, draw a line of 
progress for the most recent six dayfr of data. If the 
line of progress is flat or decelerating, make a change 
in the instructional program. Refer to the insert on 
"Using Data Decision Rules with Minimum 'Celeration" for 
instructions on drawing a new minimum celeration line* 



Minima an 



Has the student *e performance fallen below the minimum 
'celeration line for three consecutive days ? If so, 
then it is time to make a change in the instructional 
program. 




In programs designed to accelerate the duration of student 
responding, the record floor represents the smallest amount 
of time that your stopwatch can accurately record* If 
al) of the target performance is below the record floor, 
the skill is too difficult, and you should probably step 
back or slice back to an easier skill level. Example : 
In a program designed to increase the duration of working 
on task, s minimum 'celeration lin& was drawn from the 
results of a three day assessment to the program aim of 
four hours (by the end of the year). Following the initial 
assessment, 0 sec duration was recorded for three consecutive 
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days, To determine the record floor, convert the smallest 
amount of time that your stopwatch can record to a decimal 
equivalent in minutes (.1/60 « ,0016), fin-i that fraction 
on the left-hand scale of the chart, and iinart it as a 
horizontal bar across the day line. Reccrd U 0 U duration 
as a ? just below that line. Since corrects are below 
the record floor in this example, a step or slice back to 
an easier skill level should be made. 
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If this is the first phase of your program* the arwer 
to this question is no. If you have ever reached steps 
(4) or (5) on your flow sheet for the skill your student 
is currently working on, and the student's performance 
has not accelerated since, then there is a compliance 
problem. 



See the examples below if you are uncertain whether or 
not your student *s performance pattern shows high varia- 
bility or sharp deceleration in corrects. 



HIGH VARIABILITY 



SHARP DECELERATION IN CORRECTS 



lOOO r 
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To answer this question, mult^py your aim by .83.>If ^ny 
corrects are greater than .83 times the aim of your pro- 
gram, then change your program to motivate student respond- 
ing. Example : If the aim of your program is to increase 
playing with peers (e.g., minutes oi participation in group 
games) to the entire 25 minute recess period* the .83 times 
your aim would be .83 X 25 ■ 20.75 minutes. If an£ of 
the performance data is greater than 20.75 minuses, the 
answer to this question is yes » and you should turn to 
page 12 of the Handbook for suggested Strategies for fluency 
building problems. 



This refers to the line of progress for your student's 
data. If it is either flat or accelerating, then there 
is an acquisition problem in student responding. 



In programs designed to accelerate the duration of student 
responding, multiply the aim by .5 and estimate the value 
for the start of the line of prepress. If the trend of 
corrects starts at more than .5 times the aim* then you 
should change the instructional format. Example : The 
aim for the peer play program mentioned earlier was 25 
rein . Multiplying this number by .5 yields 12.5 trinutes. 
If the line of progress starts at more than 12.5 min* then 
a format problem is indicated. 



14:) 



INSTRUCTIONAL HI€RRRCHI€S RCSCRRCH PROJCCT 



University of Washington 



Norrts Harinft, Principal Investigator 



mout to 
NUT STEP 

ti) 




fctirm 



STEP AteAO 



FLUENCY 



pRoetem 



w 



DECELERATE LATENCY AND DURATION OF RESPONDING 

Experimental rules for use with minimum celeration 
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SUPPLEMENTAL RULE SHEET: DECELERATE LATENCY AND DURATION OF RESPONDING 



The Supplemental Rule Sheet is an expanded text version of the accompanying flow 
she*t for ''Decelerating Latency or Duration cf Responding" - Each diamond-shaped 
figure represents a decision point followed by a short explanation which should 
help you make an instructional decision. Since much of the information has been 
presented in the two inserts "Using Data Decision Rules" and "Data Types", a 
quick review of these sheets would ha helpful. 



*T AIM? 



If the student has met aim, you should move to the next 
curricuJar step or program as soon as possible. If the 
student has not met aim, go on to the next question. 




Is the student progressing? A student may be below the 
line but not progressing. If you think the strident may 
not be progressing, draw a line of progress for the most 
recent six days of data. If the line of progress is flat 
or accelerating, make a change in the instructional pro- 
gram. Refer to the insert on "Usxng Data Decision Rules 
with Minimum f Celeration" for instructions on drawing a 
new minimum Tceleration line. 



Has the student T s performance data been above the minimum 
'celeration l'ne for t hree consecutive days ? If so, then 
it is time to make a change in the instructional program* 



c 



If the aim of your program is to decelerate the latency 
and/or duration of student responding, then you will have 
already determined the record or performance ceiling for 
your program. This represents a n allowable time limit 
after which a contingency for no response is implemented. 
In a latency program, the performance ceiling is usually 
5 gee , whereas in programs designed to decrease the dura- 
tion of student responding (e.g., shoe-tying) the time 
limit may vary. In either case, if student responses re- 
main at the record ceiling, then you should step back to 
an easier skill level. 
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If this is che first phase of your program, the answer 
to this question is no. If you have ever reached steps 
(4) or (5) on your flow sheet for the ffkili your student 
Is currenty working on* and the student f s performance has 
not decelerated since* then there is a compliance problem 
in student responding. 




See the following examples if you are uncertain whether 
or not your student's perfonnaoce pattern shows high vari- 
ability or sharp acceleration in the target performance. 
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One method for determining high variability in the target 
performance data involves the uee of sn ordinary wooden 
pencil , Take the pencil and place it directly over the line 
of progress for the target performance data. If the majority 
of data for the line of progress can be aeen around the 
outline of the pencil, then high variability is indicated. 
In the shoe-tying program (decelerate duration) on the 
preceding page, performance haa been above the line for 
three consecutive days and a change in programming is 
needed* When a pencil is placed directly over the line 
of progress for the six most recent data, four of the six 
data points remain visible. In this case, high variability 
indicates that there is a compliance problem in this program. 




To answer this question for decelerate duration or latency 
of responding programs* multiply your aim by 1*2 . If any 
of the performance data is less than 1.2 times the aim 
of your program, then change your program to motivate stu- 
dent responding* Example : The aim for a shoe-tying program 
is 6 sec and the student performance closest to the aim 
is 1 min and 15 sec. First find the decimal equivalent 
in minutes of 6 sec, and 1 min 15 sec: 



6 
60 



,10 



60 



Multiply the aim by 1.2 to get .10 X 1*2 - .12 * Since 
student performance is not less than 1.2 times the aim, 
the answer to this decision £oint is no, and you should 
move to the next decision point on the decelerate duration 
of responding flow sheet. 




This refers to the line of progress for your student's 
data* If it is decelerating,, then student performance 
indicates an acquisition problem* 




In programs designed to decelerate the duration or latency 
of student responding, multiply the aim by 2 and estimate 
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the value for the start of the line of progress. If the 
target performance trend starts at more than twice the 
aim* then an acquisition problem is indicated. If tne 
performanq* trend starts at less than twice the aim, then 
you should change the Instructional format* Example : 
in the shoe-tying program mentioned above, the ai.d is to 
decelerate the duration of student responding to 6 sec . 
Converting this value to a decimal equivalent in minutes 
gives us .1, and multiplying this value by 2 yields .2 . 
If the line of progress for the performance data begins 
at any value less than .2 , and the student is not at 
aim, then you should change the instructional format, 
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SITE I 



Introduction 

Mr. Carl Binder provides preservice and inservice troining to teachers !n the use 
of Precision Teaching techniques, ond writes and distributes the "Data-Shoring 
Newsletter". Mr. Binder's interest in decision rules <fkJ his five year associotion with 
tne Instructional Hierarchies Research Project led to his participation in fifth yeor 
activities. Mr. Binder provided name* and addresses of teachers he bad trained in the 
Northeost, and he then distributed copies of the Handbook of Experimental Procedures 
to consenting subjects and provided follow-up in the use of the experimental decision 
rules to subjects who requested ossistonce. 

A totol of ^6 potential subjects from the New England area were invited to 
participate in the project. Mr. Binder moiled o list of names and addresses of 38 of 
the potentiol subjects to the project and they were then mailed an introductory letter, 
two copies of the consent form, (see Appendix I), o Background Questionnaire ond 
postage-paid return oddressed envelope. The Handbook of Experimental Procedures 
ond a contoct cord were moiled to Mr. Binder, who distributed these materials to 
consenting subjects. Subsequent questionnaires and inserts to the Handbook were 
moiled directly to subjects by members of the research stoff. The names of on 
additionol eight potentiol subjects were provided by their administrators 
who contacted the project directly after talking with Mr. Binder. Since these 
potentiol subjects were identified foirly late in the project year, they received the 
Handbook ot the some time they received introductory letters, ccnsent forms ond 
Background Questionnaires. Twenty-seven of the 46 potentiol subjects consented to 
participate, 10 subjects did not, nine people did not return consent f orms. (Return 
rote 80% (37/46) consent rote 73% (27/37)1. 

Subjects received no explicit training in the use of experimental procedures, 
either from Mr. Binder or project stoff, olthough ossistonce vos ovoiloble upon 
request. 



Method 



Subjects 



Subjects were 27 teachers from 1 1 different schools or residences in 
Mossochusetts, Twenty-three subjects worked in private schools or facilities serving a 
total of approximately 506 handicapped pupils. Two subjects worked in o stote school 
that served opproximotely 1200 hondicopped pupils. Two subjects worked in two public 
schools serving o totol of 47 handicapped pupils. The totol number of 
severely /profoundly hondicopped pupils served by all schools wos 1,662 ond two of the 
schools were integroted facilities serving both severely/profoundly hondicopped pupils 
and nonhondicopped pupils. Subjects served a totol of 254 pupils, including two leorning 
disabled, nine mildly handicapped, 36 moderately handicapped, 93 severely 
handicapped, 56 profoundly handicapped, 9 multiply handicapped, 8 deaf /hearing 
impaired and 30 autistic pupils. 

Twenty-six of the subjects tought 226 pupils per year, 198 on an average day, 
with o mean class size of 7.6 pupils (ronge 2-12). One of the subjects tought 28 
moderotely handicapped pupils. Teachers usually worked with groups of mixed ages 
(see Table M)> 
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TABLE l-l 



AGE GROUP OF PUPILS SERVED BY SUBJECTS 



Pupil Age Group 


No. af Pupils 


No. af Subjects 


C j___i_n_ j_n_L __ t tl \*y\ 

elementary 1 6- 1 L) 


1 Q 

1 7 


2 


Mixed Pre -school end Elementary (2-12) 


24 


2 


High So x>l (12-22) 


125 


1 1 


Mixed Elementary and High School (6-22) 


5 


1 


Above School Age (22+) 


4 


1 


Mixed High School end Older (11+) 


77 


10 


TOTAL 


254 


27 



Teachers averaged 3.8 years teaching experience iranje 1-22 years) and 2,9 years 
in special education (range 1-7 years). Twenty-one af the 27 subjects had training in 
Precision T eaching. Twenty-four subjects reported some type af assistance in their 
classrooms. One subject reported 15 assistants, six valun+eers, and twa students far a 
tatal af 476 hour, af assistance e^ch week. The ather 23 subjects reported a tatal af 
45 assistants, 4 volunteers* 6 student teachers* and 13 athers wha work a tatal af 977 
hours per week. Of these subjects, the subject with the least amount af help had four 
hours a week, while the subject with the mast help had 96 hours a week. These 23 
subjects hod an average af three helpers (range 1*5) for an average af 42.5 hours per 
week (range 4-96 haurs). The three teachers wha did nat have any assistance served 
twa, four, and ten pupils. Twenty af the 27 teachers reported that pupils in their 
classrooms received some type of therapy for approximately 5.7 hours each week 
(range 1*30 haurs). 

Subjects conducted programs on a regular basis in a variety at instructional 
settings. Twenty -twe conducted instructional programs at least once per day, twa 
subjects four times per week and three subjects three times per week, Eight subjects 
provided five or fewer trials, eleven subjects provided 10-15 trials, one subject 
provided 16-20 trials and seven provided 20 or more response opportunities. AM 
teachers requested individual responses from their pupils. Eighteen divided their 
instruction by requesting individual responses in 1:1 sessions, group sessions and 
natural occasion". Seven subjects spent at least 60% af their time in 1:1 sessions 
(mean 82%), and twa subjects spent 80% and 90% af their instructional time requesting 
pupil responses at natural occasions for responding. Subjects spent an average of 4,2 
hours per week planning (range 1*10 hours). 

Evaluation procedures used by subjects at this site included frequent data 
collection and graphing af pupil performance data. Twenty-one collected data an 
95*100% af their instructional programs, twa on 85*94%* one on 75-84%, twa on 
50*74% and one collected data on 10*24% af the instructional programs. Fourteen 
reported that they collected data every time the program was run and eleven reported 
they collected data almcst every time programs were run. One subject collected data 
about half the time programs were conducted. Fourteen subjects collected data on 
every trial, four subjects collected on special probe trials* three subjects on the last 
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trial only <rid one on o nndom number of trials. Twenty-six subjects collected rate 
dato in their classrooms, ond for 17 subjects, this was the most frequently used data 
type. Twenty-two grophed their pupils* performance data. 

All subjects utilized decision-rules in ot least some programs for deciding when 
to step ahead end what to do when o pupil met oim. Twenty-two reported rules for 
deciding when to step back in ot l-ost some of their programs* and sixteen reported 
rules far deciding when ond what instructional strotegies to change in at least some of 
their programs. 

Troining end Follow-up 

Mr. Binder octed as an intermediary between subjects and project staff in 
regords to training during the year. Subjects were free to contact Mr. Binder 
whenever ossi stance was needed on odopting the procedures to their classroom or on 
using the Handbook . 

Project s + iff assisted subjects through phone or moil contact* Assistance to 
subjects using the procedures in their classroom for a trial period was provided by 
project stoff through phone colls. 



Results 

Cast of Training 

There were no direct troining costs for subjects ot this site. The cost of mailing 
the Hondbook and other moteriats was $6.89 per subject, a total of $186,03 for the 
site. 



Evaluation of Handbook 



Site I subjects did not receive any direct training from project sta.f or the 
project intermediory, Mr. Carl Binder, on the use of the decision-rules or the 
Handbook of Experimental Procedures ( Handbook ). All 27 subjects who received 
copies of Ihe Handbook were given the opportunity to evaluate tK N moterial in the 
Hondbook through the Handbook Questionnaire. Seventeen of 27 subjects (63%) 
returned a completed Handbook questionnaire* Subjects spent differing omounts of 
time reviewing the Handbook . One subject spent one week reading the materia!, while 
the ather sixteen subjects averaged 3.1 hours (range U 10 hours). 

Eoch subject rated the clarity of information presented in rine sections of the 
Handbook on a Likert Scole ronging from 5 (Very Cleor) to I (Confusing). Some 
subjects did not respond to all items on the questionnaire. In general, there was a wide 
ronge in subjects' rotings, but most sections of the Handbook averaged "V or better 
(Table 1-2). Sheets roted sections on "Quick Reference Guides for change strategies" 
(mean kJ t range 3-5) ond "Fluency Building* 1 (mean ^.5, range 3-5) as the mos* 
understandable. 
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Eleven of 15 subjects wrote additional comment* on the overoll information 
presented in the Handboo k of Experimental Procedures . Nine of the 'I subjects 
commented on the formot of the Handbook. Three of these subjects icid that the 
overoll formot wos good (e.g., "I'm very impressed with the layout, organization, ond 
clonty of the H andbook J Two subjects said the generol formot could be in*proved 
(e.g., "You have o boatload of informotion but it's pockoged ond presented 
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awkwardly.") Four subjects said that the format af individual sections af the Handbook 
needed improvement. Twa af these four subjects cited the sections on determining 
percent correct (e.g., "With the exception af determining percent correct all 
information in the Handbook is clearly presented) or performance variability. One 
subject reported that the section on determining sharp deceleration in corrects should 
be clearer and one subject stated that the "Rules for Rate" were not clear. Twa af the 
eleven subjects commented on the contents af the Handbook. Both af these subjects 
cited the overall usefulness and importance of the information presented in the 
Handbook. 



• 



9 

ERLC 



Table 1-2 

RATINGS FOR CLARITY Or INF ORMATION PRESENTED IN THE 
HANDBOOK OF EXPERIMENTAL PROCEDURES 

Content Ratings 

mean range 

Using the decision rules (N=I7) 4.3 (3-5) 

Rules for Rate (N= 1 5) 3.7 (2-5) 

Rules for Percent Correct (N=|0) 3.0 (1-4) 

Quick Reference Guide for Change Strategies (N=I7) kJ (3-5) 

Drawing lines-af-progress, determining percent 
correct, performance variability ond sharp 

deceleration in corrects (N=l 7) 3,5 (1-5) 

Establishing initiaf instructional formats (N=I7) kJ (3-5) 

Acquisition (N= 1 7) 4.1 (1-5) 

Fluency Building (N^I7) kS (3-5) 

Compliance <N= 16) kA (3-5) 

Median Rating k3 

Adoption of Procedures 

Subjects indicated their decision whether or not to adopt the procedures and 
experimental rules in their classroom on the Trial Period Questionnaire. Eighteen of 27 
subjects (67%) returned a completed questionnaire. Of these, 1 5 of 1 8 subjects (83%) 
agreed to apply the procedures in their classrooms for a trial period. 

The three subjects who decided not to adapt the procedures each gave reasons 
for their decision. One subject was not longer employed. Another subject expressed 
continued interest and support in the project octivities, but explained that his 
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octivities no longer included direct service for handicapped pupils* The third subject 
felt that the procedures were not applicable to her students or compatible with her 
instructional procedures, since her ten severely /profoundly handicapped pupils had 
such low rates of independent responding. Since this subject indicated that her/his 
instructional ond evaluation procedures were compatible with application of the 
procedures, and since he/she rated the Handbook as very clear, this subjects 1 decision 
to not adopt the procedures may be due to a misconception of the application of rate 
data. 

Of the 15 subjects who agreed to try the procedures, seven subjects did not 
provide information to the project regarding the application of the procedures (see 
Table U3). Four of these did not octuolly use the procedures, reporting that they "did 
not have time." One subject, who became an administrator during the project, did nc; 
instruct his/her teachers in the use of the procedures as he/she had planned, Two 
subjects reported using the procedures, but did not return completed Procedures 
Questionnaires. Thus, of the original 27 subjects, 8 subjects (30%)uctually applied the 
procedures and reorted results to the project. 

The eight "yes" subjects included teachers who served a total of 52 pupils, with 
on average class size of 6.5 pupils (range 2-1 1), Two of these teachers did not have 
any assistance in their classrooms, one teacher hod 4 hours per week of help, and the 
other five teachers had a median of 36 hours per week of assistance. The nine "no" 
subjects generally had larger classes (average class size 10.5 pupils) and more 
assistance (overoge 40 hours per week) and more teaching experience (4.52 years vs. 
2.1 years for "yes" subjects). Instruction and evaluation methods generally did not 
differ between the two groups, although five of the 9 "ho" subjects did not graph their 
data. The "no" subjects spent less time reviewing the Handbook and rated it lower 
(average 4.0) than the 'Ves* 1 subjects (average Handbook rating of 4.7). 




Toble U3 

COST OF TRAINING AND FOLLOW-UP (N = 1 5)* 



Subject ft 


Agreed to 

Trinl Period 

I 1 Ivl * tl 1 


Requested 
Fnl low-im 


Received 

Pnl 1 AW * 1 tO 


Cost of . 
FoMow*t JO 


Totol Cost of 

Trnlninn f nl Intuit if* 


Reported 

I Ut& nf Prnr** 
V3C vi n vet 


5301 


x 

f\ 


x 


x 


513 04 


519 93 


no 


5302 

J *J \J c 


Y 


x 


Y 

/v 


< 1 g 99 


523 88 




5314 


x 

f\ 


x 




5 0 00 


5 6 89 


no 


5321 


x 


x 


x 


5 3 37 


510 26 




5326 


x 


x 


x 


Al5 86 


522 75 


nn 

1 IV 


5327 


x 


nn 
t tu 


nn 
i iu 


< 0 00 


5 6 89 




5328 


x 


x 


x 


$ 2.28 


5 9. 17 


nn 

1 IV 




Y 

A 


x 


nn 


5 0 00 


5 6 89 


nn 
i ly 


*i336 


Y 

A 


x 


x 


5 9 46 


516 35 




533R 


x 


nn 

1 IV 


nn 


$ 0.00 


$ 6.89 


no 


5339 


X 


X 


V* 

/\ 


$ 5.82 


$12.71 


yes 


534i 


X 


X 


no 


$ 0.00 


$ 6.89 


no 


5342 


X 


X 


no 


$ 0.00 


$ 6.89 


ytj 


5346 


X 


no 


no 


$ 0.00 


$ 6.89 


yes 


5347 


X 


X 


X 


$15.47 


$22.36 


yes 



Includes long dssfunce charges, 
* Three subject* <i:d not ogree to use procedures 
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Faltow-up 

Eight of the eighteen subjects returning Trial Period Questionnaires 

requested follow-up assistance in the Application of the procedures. Of these eight, 
four actually received technical information. Twa of these four eventually returned 
Procedure Questionnaires (Table 1-3). Each af the other four subjects were contacted 
by telephone, but, since no questions were asked by the subjects, no technical follow- 
up was provided. Of these four people, only one returned a Procedure Questionnaire. 

Four other subjects requested follow-up assistance during the application of the 
procedures. A total of 12 of the 1 8 subjects who returned a Trial Period Questionnaire 
eventually requested fallow-up (67%), eight received technical information c*id seven 
reported on use of the procedures (Table 1-3). 

Content of the follow-up information included assistance on specific 
instructional programs (2 subjects), setting aims (2 subjects) md use of the flow chart 
(2 subjects). Other content included: punishment, noncompliance, determining 
percent correct, drawing minimum tereration lines, use of physical assists, component 
vs. whole task techniques, deceleration of errors during fluency building, use of the 
rate plotter, use of Decision Record Sfrieets, and the application of changing aim 
strategies. 
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Twelve subjee+s requested folL v-up and eight subjects received specific follow- 
up information (Table 1-3). One subject was provided information in four separate 
phone calls, one subject in two calls, and one subject in one letter and one call; all 
other subjects were follaw-up in a single phone call (total follow-ups: 12 phone calls 
and I letter?, The total cast of the fallow-up to eight subjects was $91.75. The cast 
per subject ranged from $2.28 - 16.99 with a median of $1 1,25. Of the eight subjects 
who received follow-up, three did not use the procedures; tatal cost for these subjects 
was $40.64* Thus, total follow-up cost for subjects wtia did use the procedures was 
$51.11 (Table I- 10). 

Applica tio n of Procedures 

Thv eight subjects app!ied the experimental procedures over a total af 69 weeks 
in 51 instructional programs far 13 different pupils, 27% of the pupils served by these 
teachers. Five teachers, each af whom taught six pupils, applied the procedures in one 
or more programs tatal 27) for one pupil each. Subject 5327, with nine pupils, applied 
the procedure; to three programs for twa pupils* Subject 5321 applied the procedures 
in 6 programs for four of 1 1 pupils, and Subject 5302 used decisionaries with both af 
her/his pupils in a total af 15 programs* 

Previously, five subjects indicated that they preferred to codec' Jte data and 
three subjects indicated that they preferred to collect accuracy data or counts af 
behavior* nevertheless, rate data were collected in every program in which the 
procedures were applied (regular rate in kk programs, adjusted rate in 7). Six subjects 
applied the rules using the minimum 'celeration procedure, and twa subjects used the 
method of drawing I ines-of -progress every six dcys and comparing the patterns* 

The usefulness of the Handbook to subjects during the application of the 
procedures was given a median average rating of 4.25 (Table 1-4), as compared to an 
overall rating of 4.44 prior to application* Changes in ratings during use on" the rules 
are shswn in Figure S-l. Subjects rated the sections of the Handbook c jncernjng 
technical information necessary for application lower thon section dealing with 
instructional strategies and design, 
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Table M 
USEFULNESS OF THE HANDBOOK 
DURING APPLICATION OF THE PROCEDURES 



Topic Rating Mean Range 



1. 


How ta Make on Instructional Decision (N=7) 




4-5* 


2. 


Quick Reference Guides for Change Strategies (N=3) 


i».5 


2-5 


3. 


Drawing Ltnes-af-Progress (N=8) 


3.75 


2-5 


i*. 


Determining Percent Correct ((N=8) 


2.63 


1-4 


5. 


Determfng Performance Variability (N=8) 


3.13 


1-5 


6. 


Determining Sharp Deceleration in Corrects (N=8) 


2.73 


2-5 


7. 


General Considerations for Instructional Formats (N=y) 


i.S 


4-5 


8. 


Acquisition (N=8) 


i».5 


3-5 


9. 


Fluency-Building (N=8) 


it. 5 


3-5 


10. 


Compliance (N=8) 


4.25 


2-5 


II. 


Median Rating (N-8) 


i».25 


3-5 
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Since the technical sections were rated lower, and since na direct training was 
provided ta these subjects, it is nat surprising that the subjects generally had some 
difficulty in the correct application of the procedures. Five subjects shared pupil 
performance data and evaluation af their application af the procedures showed several 
major errors! mast commonly, subjects did nat draw new minimum 'celeration lines 
following a phase changes and did nat moke changes when the pupil's ^rformance fell 
below the ninimum 'celeration line. 

Prior ta the use af the procedures, subjects reported planning for a median af 4,0 
hours per week (range UIO); during the application af the procedures, the median was 
3.5 hours per week (range L5). Four subjects felt that their planning time increased 
during application of the procedures, and four felt that it remained the same- in two 
cases, planning time octually increased, in one case planning time remained the same, 
and in five cases, the reported planning time actually decreased (Figure 1-2), 
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Subjects rated their attitude toward the procedures as 4.63 overall (Table 1-5). 
Uue to a researcn error, no ratings af their attitude prior ta application af the 
procedures were made. These subjects, with previous training in Precision Teaching, 
gave the highest ratings to collecting and charting pupil performance data. 

Table 1-5 

SUBJECTS' RATINGS OF ATTITUDE TOWARDS THE PROCEDURES 
DURING THE TRIAL PERIOD 

Topic Ratings (N = 8) 

Mean Range 



1. 


Using the procedures when instructional 
programs are established 


4. 37 


3-5 


2. 


Collecting performance data 


5.00 




3. 


Charting performance data 


5.00 




4. 


Drawing Minimum 'celeration Lines 


4. 43 


3-5* 


5. 


Drawing Imes-af-progress in order 
ta apply the rules 


4. 00 


4-5 


6. 


Using the expenmental decision rules 


4. 50 


4-5 


7. 


Using the change str jtegies for 
instructional programs 


4.38 


3-5 


8. 


Overall 


A. 63 


4-5 



N = 7Subjects J^J ; 
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HOURS PLANNING BEFORE AND DURING APPLICATION OF DECISION RULES 
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Impoct of Fopil Performonce 

Seven subjects estimoted that the decision-rules occeleroted pupil performance, 
while one subject felt they mode no difference. Seven of these subjects p)on on 
continuing to use the procedures. The subject who estimoted that the procedures 
mode no difference wos using the *drow patterns" procedure for application of the 
decision rules. The other subject using the "draw patterns 11 method does not plon on 
continuing to use the procedures* olthoogh he/she estimotes thot they occeleroted 
pupil performance, 

Five subjects provided pupil performonce dato for nine different pupils working 
on 31 different instructional progroms. The instructional progroms were oimed ot o 
wide range of behaviors. The tcrgeted movement cycles and the age ronges of the 
pupils as reported by the subjects ore shown in Toble 1-6, 

Tobte 1-6 

TARGETED MOVEMENT CYCLES IN PUPIL PERFORMANCE DATA 

Age Range of Pupils Movement Cycles* 

Writes digits) writes digits to answer multiplication focts; 
writes nome; soys three/four letter wordf 

13- 19 Wntes digits for oddition to 100; writes digits to answer 

subtraction focts; writes nome; writes copitol letters* 
writes digits to answer moth focts 

14- 23 Ploces cup {right & left hands); releoses carrot (right & 

left honds) 

26-27 Lifts knee; tokes steps* counts coins; soys I- 100; swings 

arm; counts by 5 ! s; soys doys of the week; counts dots in 
orroy; counts three objects; flexes ankle; bends knee; 
counts numbers 



* Movement cycles ore given generollv as described by subjects; most subjects 
counted both correct and error responses, although tnat distinction is not mode obove. 
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Each of the subjects w K o provided pupil performonce doto used the minimum 
^ceterotlan line method for deiermining when on intervention should be made, olthough 
there were many errors in the application of this procedure. There were instonces 
where subjects did not drow new minimum ^elerotion lines following the changes. In 
phases where there were minimum Velerotion lines, pupil performance fell below the 
line for three consecutive doys on 32 occosions. Less than half the time (15), subjects 
then mode a change; in the majority of these "intervention opportunities 11 the subject 
continued with the ineffective procedures. Subjects olso chonged instructional 
strotegies thot were effectively focititoting pupil progress cn four occosions. 
Sometimes subjects did not indicote performonce oims on the chort. The lock of oims 
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on the cnart may indicate the lock af any aim for performance; in any case, the pupils 
af four subjects (atl except those af subject 5302) shewed almost na progress through 
the curriculum, with one "step ahead in curriculum' 1 for seven pupils in 16 programs in 
723 calendar days (Table 1-7). 

These four subjects alsa mode relatively few interventions, 13 in 340 days af 
data. Three af the subjects used suggested strategies (decision in occord with the 
rules) everytime they used an intervention, while Subject 5302 fallowed *he rules in 
53% af the interventions (Table U8). One subject submitted data that could nat be 
analyzed. 

Eoch subject wha submitted pupil performance data estimated that the 
experimental procedures accelerated pupil progress. The percentage af successful 
interventions ra tged from 50% ta 100%. Overall, 70% af the interventions were 
successful, and 50% af the interventions were made in accord with the rule 
procedures. Subject 5321 submitted pupil performance data covering before and 
during the application af the procedures (Table 1-9). The six programs with four 
pupils, the teacher stepped ahead three times but made na strategy changes prior ta 
the use af the experimental rules. During the application af the rules, the teacher 
stepped dhead once and made six interventions. 

* 

Accuracy of the Rul es 

Notations made by the subjects af the types af intervention strategy changes 
they made were used ta determine whether the change was Vi accord with the 
intervention strategies suggested by the rules. In twa cases, the description provided 
by the subject was unclear, and these cases were eliminated from analyses. There 
were a total 321 changes which were analyzed. The accuracy af the rule in predicting 
pupil progress was 65% (Table UI0). 
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Table 1-7 

ANALYsiS OF PUPIL PERFORMANCE DATA 



Subject 


Pupils 


Programs 


Calendar 
Days 


Data 
Days 


Phases 


Mep 
Aheads 


Inter- 
ventions 


Usable 


^Successful 


%Successful 


Subject's 
Estimote of 

Impact of 
Procedures 


5302 


2 


15 


1767 


668 


71 


13 


29 


22 


15 


68% 


Accelerated 


5321 


4 


6 


267 


134 


15 


1 


6 


6 


5 


83% 


Accelerated 


5339 


1 


4 


214 


122 


8 


0 


6 


4 


2 


50% 


Accelerated 


5346 


f 


4 


132 


55 


5 


0 


1 


1 


1 


100% 


Accelerated 


5347 


1 


2 


110 


29 


3 


0 


0 


0 






Accelerated 



TOTAL 9 31 1757 739 102 13 58 34 24 71% * 



170 



Table 1-8 

UTILIZATION OF SUGGESTED STRATEGIES 



Subject 


// interventions 


ff usable 


9 Successful 


ib Successful 


# Using Suggested 
Strategy 


% Using Suggested 
Strategy 


Predictability 
of Rules 


53^? 


28 


22 


15 


68% 


8 


53% 


60% 


5321 


6 


6 


5 


83% 


6 


100% 


83% 


533? 


6 


4 


2 


50% 


4 


100% 


50% 


5346 


1 


1 


1 


100% 


1 


100% 


100% 


5347 


i 


0 












TOTAL 


43 


33 


23 


70% 


1* 


58% 


65% 
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Table 1-9 

INDIVIDUAL PUPIL PROGRESS BEFORE AND DURING APPLICATION OF EXPERIMENTAL RULES 



Before Use of the Rules 



During Use of Rules 



Subject Pupil Behavior Data Step Av. Days Inter- % Success Data Step Av. Days Inter- % Success % Follow 

Days Aheads To Step ventions Interven- Days Aheads To Step vent- Interverv- nules 
Ahead tions Ahead ions ions 



5321 



writes 
digfts 

writes 

mult* 

fnrrts 



38 
25 



38 



says three 31 

letter 

words 



wntes 
name 

writes 
digits 

writes 

mult, 

facts 



30 



31 



0 
0 



0 
0 
0 



13 0 



7 l 



4_ T_F 1 1 



i6 0 



12 0 



26 0 



100% 
!00% 

0% 

100% 
100% 
100% 



100% 

100% 

100% 

100% 
100% 
10^% 
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Table I - 1 0 

ACCURACY OF RULES FOR INTERVENTION CHANGES 



Changes made in accord with a rule 
(Prediciton That progress will be 
accelerated) 



Pupil Progress 
Actually Accelerated 



Pupil Progress 

Actually_pid Not Accelerate 



48%* 
(N=I5) 



13% 



Changes made lot in accord with a 
rule (Prediction that progress will 
not be accelerated) 



23% 
<N=7) 



16%* 

(N=5) 



♦Concurrence of prediction ond actual results indicates ru'e accuracy: total 20/31-65%. 
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Cost Analysis . The totol cost for providing Handbooks and follow-up to 27 subjects in 
this site ore shown in Toble l-ll. Of the 27 subjects, eight octualty utilized the 
procedures in their clossrooms for o total of 13 pupils over 91 weeks. If all site costs 
ore included, t!,2se figures result in on averoge per pupil cost of $ 1 0*29 during the 
experimental period of opplicotion, or approximataly $3.05 per week. 

If costs are separated to include only those costs incurred by subjects who 
utilized the procedjies, these figures are reduced to o median cost of $6.89 per pupil 
and $1.33 per week (Table Ul 2h For subjects who opplied the procedures, the cost per 
program per week averoged $.13. Figures for each subject vory considerably (Table i- 
12), and would have been considerably lower if subjects had involved more pupils in 
their applicotion of the experimental procedures. 



Table I- It 



COST SUMMARY 



No. Subjects Participotmg 
Total Hondbook Costs 
Toial Follow-up Cost 
TOTAL SITE COST 




Av. Per Subject Cost 
Av, Pftr pupH Cost* 
Av. Per Week Cost** 



$ 10.29 




* Includes N= 1 3 pupils whose teochers opr lied the procedures in instructional progroms 
## Includes N=9I weeks during which teochers opplied procedures 



Table 1-12 

COSTS FOR SUBJECTS APPLYING PROCEDURES DURING EXPFR MENTAL PERIOD 



Application of Proced*j»cS 



Costs 



Cost Analysis 



Subject Weeks 


Pupils 


Program* 


Handboc! ' 


Follow-up 


Total 


f>er Pupil 


Per Program 


Per Program/Week 


5302 


13 


2 


15 


$ 6,89 


$16,99 


$ 23,88 


$11,94 


$ 1,59 


$1 ,33 


,09 


5321 


8 


U 


6 


6.89 


3,37 


,r i,26 


2,57 


1,71 


1,28 


,21 


5327 


6 


2 


3 


6,89 


0 


6,39 


2.4$ 


2,30 


1.15 


.38 


5i36 


6 




? 


6,85 


9.46 


16,35 


16, 35 


2,34 


2.04 


.29 


5339 


6 




2 


6,r; 


5,82 


12.71 


12.71 


6,35 


2. 12 


1,06 


53<(2 


1 1 




10 


6,89 


0 


6.89 


6.89 


,69 


.63 


.06 


S3U6 


6 




4 


6.89 


0 


6.89 


6.89 


l,/2 


1,15 


,2<> 


53<(7 


6 




2 


6,89 


15,47 


22.36 


22,36 


It, 18 


3,73 


1 .8.'> 


TOTAL 

MEDIANS 


91 


13 


V) 


$55-1? 


$51,11 


$106.23 
I II. $8 


$ 


$ 1,59 




$1.33 
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SITE 2 
Introduction 

The subjects from Site 2 were given the opportuniT ^ pu: .pate in J he project 
through a cooperative arrangement between the Cent r for Inservice Training and 
Program Development (CiTPD) ond the Instructional Hierarchies Research Project. 
The CITPD provides services for severely and profoundly handicapped students and 
their families through a demonsf ration center and training programs. The training 
activities include two three-week Summer Workshops , three one-day mini -workshops 
and fo!low-up cl, stance to trainees, with a major content emphasis on systematic 
instruction and the integration af services through a tr comprehensive education team 1 * 
(See Appendix 2 for a more compiete description of CITPD). CITPD identifies 
prospective schools and districts throughout the Western Region for participation in 
tl»e training program. Special education directors for these districts are sent 
information describing CITPD and gfven the opportunity to send a team from their 
district for training. 

Prior jo the two workshops, all prospective stbjects identified by ClTPD were 
sent: a letter from the !H project describing the project and inviting their 
participation; a letter from the ClTPD supporting the research project; two copies of 
*he Site 2 consent forms; thf Background Questionnaire, and a postpaid return- 
oddressed envelope (Appendix 2). 

Forty-eight consent forms aid Background Questionnaires were mailed to 
participants of the June and July workshops. Twenty-seven consent forms were 
returned prior to the workshops Consent forms and Background Questionnaires were 
also available to participants at both workshops prior to training in the experimental 
procedures. Twenty-one participants of tKe June workshop agreed tc ser v e as 
subjects, four did not consent anu o^e did not return the form or attend the wcrkshop. 
Sixteen pcrticioants at the July workshop agreed to serve as subjects, three did not 
consent and three did not return the form or attend t!ie workshop. A total of 37 
participants agreed to be subjects. 

As part of the cooperative Agreement, the IH Project agreed to provide training 
to CITPD staff members in data collection, charting, and the experimental decision 
rules prior to the June and July workshops and to conduct a training session for all 
workshop participants (not just consenting subjects) at the two workshops on the 
experimental procedures. Although CITPD staff had planned to train oH participants 
in the use, collection, and charting of different data types, this training was provided 
IH staff. 

During the workshop, all participants were required to attend "general** sessions 
and practice and free to choose among a variety of "menu" sessions (see agenda in 
Appendix 2). Training provided by the lh project at both the June and July workshops 
consisted of a large group "generol* session and two "menu" sessions. All war! shop 



Workshops were conducted by the Center for Inservice Training and Program 
Development (OTPD) and sponsored by the Washington State Office of Public 
Instruction; the Certer for'the Severely Handicapped and the Western Region Inservice 
Training Project (two projects funded through the Bureau of Education of the 
Handicapped, Department of Education), and the Uniwsity of Washington, College of 
Education, Area of Special Education. 

1 .v> 
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participants attended the general session, which was Q ,one hour presentation titled 
"Decision Rules: Wha[ l to Change" and consisted of an overview of the experimental 
procedures. Two "menu" sessions, titled "Data Decisions" and "Instruct ioncl 
Procedures", each Ifr hours long, were offered at both workshops. Thirty-one subjects 
atter>ded the sessions offered by the tH staff; 17 in June and 14 in July. 

Method 

Subjects 

Subjects who consented to project participation wer'i from 14 schools in nine 
school districts in the states of Louisiana, Texas and Washington. The schools included 
10 public schools, one sta'.e school, two state institutions and one private residential 
school. Seven of the schools served both nonhandicapped and handicapped pupils, 
ranging from mildly handicapped to*profoundly handicapped. The other seven schools 
served handicapped pupils exclusive!/. 

Twent^^three of the 37 subjects were classroom teachers. One teacher also had 
some administrotive duties. Twelve subjects were therapists, including four physical 
theropjsts, four occupotional therapists, and four communication disorders specialists. 
Although administrators were not originally considered as potential subjects, two 
subjects expressed an interest in learning the procedures in order to teach the 13 
professionals they supervised, and were included as subjects. 

The twenty-three teochers served about 220 pupils per year. Twenty-two 
teachers served a total of 1 78 pupils on an average day, with an average class size af 
8.1 pupils (range 5-14), and approximately 200 pupils per year. One subject served 20 
learning disabled pupils In a resource room. Subjects served pupifs with a variety of 
handicapp'ng conditions (see Table 2-1), although the majont/ were moderately or 
severely handicapped. 



Table 2-1 

PUPIL > SERVED BY CONSENTING SUBJECTS: SITE 2 



Subjects 


Learning 

Disabled 


Mildlv 
Handicapped 


Moderately 
Handicapped 


Severely 
Handicapped 


Profoundly 
Handicapped 


Multiply 
Handicapped 


Other* 


Total 

Pupils 




Teachers 
N=22 


10 


25 


33 


85 


42 


7 


2 


204 


Therapists 
N--I2 


V) 




118 


CO 


46 


4 


76 


382 


Total 

N=3'* 


39 


7'4 


151 


145 


88 


II 


78 


586 





^Communication rlisorders (70) Orthoperlically handicnpperf (8) 
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An overage of 35 preschool -aged pupils were served in five classrooms, 61 
elementary pupils (aged 6-1? vears) were served in six classrooms and 76 high school 
students were in seven classrooms* Three classrooms also served a total of 30 pupils in 
mixed age groups of 6 to 22 years, and two classrooms served 18 students older than 22 
years* 

Therapists served a total of 405 pupils per year and 148 on an average day, 
ranging from 2 years to older than 22 years. The mean number of pupils served by □ 
therapist per day wjs 12*3 Irange 5-20)* 

Twenty-two teachers had previous teaching experience (mean 5*8 years, range 
1-28 ^Sars) and all had experience in special education (mean 5 years, range 1-21 
years) * T herapists were slightly more experienced than the teachers* Arean number 
of years in education for therapists was 7*2 years (range 1-23 years), and 6*9 years in 
special education (range 1-23 years)* The two administrators each reported having 
supervised professionals serving sph pupils for one year* Formal training in Precision 
Teaching was reported by four teachers, one therapist and one odmin ; strator* 

Twenty-one teachers and eight therapists had some assistance in their 
classrooms* These s*„\jects reported a total of 35 paid assistants, 20 volunteers, 34 
student teachers and 50 other people, for a total of 1213 hours each week* The two 
teachers who reported the most help had help for 85 hours per week, while the teacher 
with the least amount of he p received II hours of assistance per week* These 21 
teachers had an average of 5*7 persons providing some tyoe of he!p for an averoge of 
43 hours per week, and all teochers reported t!iat their pupils received some form of 
therapy in oddition to regular classroom activities. Eight therapists had assistance. 
The average number of assistants for each therapist was 2*4 for an average of 38.6 
hours per week (range 7-20). Neither of the administrators reported having assistants. 

There were mC;iy points of similarity in the instruction provided by the teachers. 
Eighteen teachers ran instructional programs o* '***t once per day* Eight teachers ran 
progrcms less often: two teachers ran instructional programs four times a week, four 
teachers ran programs three times a week, one teacher twice a week, and another 
teacher ran programs whenever o natural occasion for the behavior occurred* Six 
teachers provided pupils with five or fewer response opportunities, five provided five 
to nine t r ials and ten provided 10-15 trials. 

While alt teachers reported spending some instructional time requesting 
individual responses in a 1:1 setting, nine teachers spent the majority of their 
instructional time in a !:1 setting. Two teachers spent 50% or more of their 
instructional time requesting individual responses from pupils in a group setting, and 
two teachers spent 50% or more of their instructional time requesting individual 
responses at natural opportunities* Two teachers spent at least 50% of their 
instructional time requesting unison responses in group settings and one teacher 
reported spending the majority of her/his time doing feeding and mafntenance tasks* 
Six other teachers diviaed thefr time equally between two instructional settings* 
Three of these teachers spent c*> equal amount of fime requesting individual responses 
in group settings and at naturoi opportunities, two spent equal time requesting 
individual responses during I;! instruction and group settings and another teacher 
divided her/his tin ( e equally between inaividuct responses in 1:1 settings and at natural 



One teacher had no previous teoching experience and did not complete parts 3 and 4 
of the Background Questionnaire. 
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opportunities* The overoge planning time was 4*5 hours per week Cange UI2) except 
for one subject, who reported spending 20 hours per week. 

Instruction conducted by therapists differed from that of the teachers. 
Therapists generally conducted their instructional programs less frequently than the 
teachers: four therapists ran programs once or twice per week and six therapists ran 
programs once, twice or three times per week* Only two therapists ran programs once 
or twice per day* Three therapists provided pupils with five or fewer response 
opportunities, three provided five to nine trials, four 10-15 trials and two 20 or more 
trials* 

■Jke the teachers, all of the therapist* spent some of their instructional time 
requesting individual responses in a It I setting* Nine therapists spent 50% or more of 
their time working with pupils in a 1:1 setting and one therapist spent 50% or more of 
his/her time requesting individual responses in a group setting* Two therapists divide 
their time between requesting individual responses during 1:1 instruction and group 
settings* The therapists spent an average of 2*6 hours per week planning instructional 
programs (range 1-8 hours), which was considerably less time than the teachers 
reported. 

Rfteen of 2? teachers collected data on at least 75% of their programs (medion 
category was 65-94%)* Three teachers collected data on 50-74% of their programs, 
two on 25-49% and two on less than 10%* Nineteen teachers collected data every time 
or about every time a program was run, and three teachers collected data about half 
the time tK e program was run. Eighteen teachers collected data on every trial, three 
tn a random number of trials and one on a!! trials during u set amo:nt of time. 

The most frequent data type collected was accuracy data, with 16 teachers 
rank 7 ng it first* Five teachers selected count of behaviors as their preferred data type 
and onother teachc selected count of trials* Of the eight teachers who graphed data, 
two had formal training in Precision Teaching* The majority of teachers had specific 
rules for moving to the next step and changing when ar aim for all or some of their 
Progroms* Four teachers rer Tted rules for when to change instructional procedures 
and three had rules for sefec ng intervention techniques all programs. 

Seven therapists collected data op at least 85% of their instructional programs, 
two therapists collected data on 75-84% of their programs, cie on 50-74%, and two 
therapists collected data on less than 24% of their instructional programs. Six 
therapists collected data every time or ahout every time a program was run. One 
therapist collected data about KtVf the time a program was run, and four therapists 
collected data only for initial assessment purposes* One therapist collected data every 
time a program was run with a severely/profoundly handicapped pupil, about every 
time a program was run with a moderately handicaped ptpil, and collected only initial 
assessment data when working with mildly handicapped pupils* 

Three therapists collected data on every trial, one therapist collects data only on 
the first trial, three collect data on a random number cf trials, three on special 
"probe" trials and mother therapist collected data on all trials during a set amount of 
time. One therapist collected data on every U ial for programs run with 
severely/profoundly handicapped pupils and on specie! **pr6be" trials only for programs 
run with moderate and mildly handicapped students. 

Fou therapists collect count of behavior data. Three other therapists collected 
accuracy data most frequently, two collected levels of assistance mosl frequently, one 
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count of trials, and one collected rate data most frequently* Uf>e therapist collected 
measures of pulmonary functioning* 

Like the teachers, the majority of therapists reported specific rules for moving 
to the next step (N= I I) and changfng when at aim (N=I0) fo> at least some programs. 
Only one therapist reported rules for deciding how to modify instructional procedures 
or when to change the instructional strategies during a program* 

Training Procedures 

All workshop participants, including the subjects, attet.Jed □ one hoor general 
session, "Data Decision Rules," which covered the history of the IH Project, the need 
tor decision-rules, prerequisites for using the rules and examples of rule application to 
pupil performance data* Since CITPD empho^ized the use of percent correct data, 
these data and rules were presented : n the general session* (Rules utilized the 
"drawing patterns" opproach, os detailed in the General Procedures Section!. 
Participants were taught how to draw lines-of-progress and apply the rules to percent 
correct data, the data type promoted during the CITPD sessions* Participants 
practiced drawing patterns and applying the rules, ond received handouts on Mr >ules for 
Percent Data", drawing lines-of-progress for six data days ond examples o» charted 
data for practice* 

Subjects were provided additional training in two "menu" sessions, each offered 
twice at each workshop. The first of these session^ "Data Decisions," was Ik hours 
long* Groups of five, twelve, six and eight subjects attended those sessions (Nr3l). 
(Six subjects dropped from the project at this time*) The content of this session 
included an introduction to different data types, methods for collecting and charting 
rate, latency, and duration data, and an introduction to using rules with rate data. 
Subject^ proctlced drawing lines-of-pr ogress for five and seven data days and using the 
rules with examples of charted data* 

The second "menu" session, "Instructional Procedures", olso Ifc hours long, was 
attended by groups of eight, nine, eight and six subjects* Content Included 
descriptions of instructional strategies for initial instructional formats for both 
acquisition and fluency-building, ond strategies appropriate to acquisition problems, 
fluency-building problems, and compliance problems. At the conclusion of this third 
sessici, each subject was csked to evaluate the training (Presentation Questionnaire). 

All 31 subjects then received the Handbook of Experimental Procedures* 

^ i 

' . i 

Revisions and additions to the Handbook were^qjailed to th(e subjects during the 
fifth project year. These included the modified rules for uste with the minimum 
■celeration procedure for rate, duration and latency data, and a vr^tten description of 
how to collect and chart these data types* 

Follow-up 

rol.ow-up to Site 2 subjects was provided by both the IH staff and CITPD staff 
during the 1979-80 school year* The IH Project follow-up was provided * s ough 
telephone and mail by members of the research staff* Throughout their involvement 
with the project, each subject was enco 1 "raged to call (collect) wherever any questions, 
concerns or problems arose. Subjects were called at their request or to prompt return 
of project questionnaires* 
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Fo) low-up provided by CITPD was determined by o contract completed p^or to 
participation in The training* The contract consisted of a set of objectives and a time* 
line by which the team members of txich district agreed to rfteet the objectives* A 
trainer from the CITPD was assigned to coordi.iate follow-up assistance to each team* 
This technical assistance consisted of a minimum of six, one-day visits io each team, 
inservice presentations, and telephone contacts as requested by the subj*cts* in 
contrast to the follow-up assistance provided by the IH proiect, the CITPD staff 
assisted mainly by direct contact wi*h *he subjects* The assistance varied from small 
group discussions to individual meetings be*ween the trainer and the subject, 

Data Collection 

Changes were made in the Presentation Response Questionnaire for subjects at 
both workshops* Two questions were changed or, the Presentation Response 
Questionnaire for subjects at *ne June workshop: Question 3, ^art I, wos deleted 
because all information on charting data was presented by CITPD* In Question 5, Part 
I, "lines-of-progress" replaced "patterns". Items of the Presentation Response 
Questionnaire were reorganized for subjects nttcnding the July workshop, but the 
content remained unchanged. The new questionnaire was similar to the order in which 
the information wos presented at the IH sessions* 

Results 

Evaluation of Training 

Thirty-one subjects completed a Presentation Questionnaire following 
attendance at three training sessions by the IH staff* T /enty teachers, nine Ih^rapls's 
and two administrators commoted the Presentation Questionnaire. 

The overall rating on clarity of the information presented wis high, wirii /he 
therapists' rating slightly hiyher (mean 4.8, range 3-5) than the eachers 1 (mean 
range 3-5)* The odministratcrs* ratings were 4 ond 5, 

The subjects also rated the clnr^y of information presented on specific cs; 
establishing initial instructional fornmts; collecting datjj drawing lines-a* -progress; 
decision rules; typer of changes in instructional procedures, and using the Handooc ; 
(Table 2-2)* Ratings for the teachers, therapists ond odministra*ors were very similar* 
In jnly two cases were ratings below 3* Informction on collecting data One Workshop 
subjects only) and drawing tines-of -progress were considered ta be the -fiost c!earl/ 
presented information, white information presented only at the July Workshop on 
collecting and charting data was consdered to be the leaat clearly presented 
information* 

Subjects who had had previous training wi*h Precision Teaching Techniques (N=*0 
did not tend to rate the clarity of if <mation nigner than thas^ who had not had this 
training* On the other harvd, those Subjects who generally used specific criteria for 
modifying insracMonu! programs (N*2l) rated the clarity higher in ill categories than 
did those subjects who ysea few or no rules. 

Sixteen subjects wrote commerV"* on the training procedures* The^e subjects 
most frequently wrote that they considered the information on the experirnentot 
procedures to be very votuable und practical, and that the presentation fornat was 
goou or\<f well organized. Two tejchers express^ some concern aboc't the ■ uwnt of 
new information presented. >ne teacher requested more examples and three 
therapists suggested allowing more time for presenting the muter 
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Toble 20 

SUBJECTS' RATINGS OF CLARITY OF PRESENTATIONS 

Presentation Topic Teacher Rating Therapist Rating Admnistrator Rating All Subjects 

N=20 N=9 N=2 N=3I 



mean range mean ranqe mean range 

General purpose 4.3 (3-5) 4.4 (4-5) 4,4 4.2 (3-5) 
of procedures 

Collecting data* 4.7 (4-5) 4.7 (4-5) 4.7 (4- 5) 

Collecting and 4.3 (4-5) 4,5 2.4 3.9 (2-5) 
charting data** 

Drawing lines-af- 4.6 (3-5) 5.0 3,4 4.2 (3-5) 
progress 

Using decision- 4.4 (3-5) 4.3 (3-5) 4,5 4.5 (3-5) 
rules 

Establishing 4.3 (3-5) 4.3 (3-5) 4,5 4.4 (3-5) 

instructional 

formats 

Instructional 4.. (3-5) 4.3 (3-5) 4,5 4.4 (3-5) 
procedures 

Implementing 4.2 (3-5) 4,4 4, 5 4.2 (3-5) 

instructional 

strategies** 

Using the 3.8 (2-5) 4.0 (3-5) 3,5 4.0 (2-5) 
Handbook 

Overall 4.4 (3-5) 4.8 (3-5) 4, 5 4.5 (3-5) 

*June Workshop participants only; teachers, N=I0; therapists, N=7 

**July Workshop participants only: teachers, N=IQ; theropists, N=2; odminst rotors, N=2 
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Attitude Following Training 

Subjects rated their general attitude towards the experimental procedures 
following training (Table 2-3). Ratings of the 20 teachers and nine therapists were the 
same (mean 4.4, range 3-5 and 4*5, respectively). Both administrators rated their 
attitude as 4. Teochers rated the usefulness of the procedures at 4.6 (range 3-5) and 
the therapists ratings were slight!/ higher (mean 4.8, range 4*5). Both administrators 
rated the usefulness of the procedures as 4. 

Subjects also rated the material presented on how applicable the procedures ma/ 
be to their particular educational setting (Table 2-3). 

The subjects 1 ratings of the applicability of material were compared on a number 
of variables. The frequency with which subjects ran programs did not appear to be 
related to how subjects rated the applicability of the material. However, subjects who 
collected data almost every time a program was run, or who collected data on at least 
95% of their instructional programs, rated the applicability higher than those subjects 
who collected data less frequently. Subjects who charted their pupil performance data 
rated the applicability of the Information on collecting and charting data higher than 
those subjects who do not chart performance data. 
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Table 2-3 

SUBJECTS' ATTITUDE TOWARD THE EXPERIMENTAL PROCEDURES 



Teacher Rating 
N=20 



Therapist Rating 
N=9 



Admnistrator Rating 
N=2 



All Subjects 
N=3I 



General Attitude 

Usefulness of Procedures 

Applicability* 
Overall 

Designing initial 

instructional 

formats 

Conducting 

instructional 

programs 

Collecting data 

Charting data 

Using decision- 
rules 

Chonying 

instructional 

methods 



mean range 

4.4 (3-5) 
4.6 (3-5) 

4.5 (3-5) 
4.3 (3-5) 

4. 5 (3-5) 



4.6 
4.5 
4.7 



(3-5) 
(3-5) 
(3-5) 



4.7 (4-5) 



mean range 

4.4 (4-5) 
4.8 (4-5) 

4.2 (3-5) 

4.3 (3-5) 

3.5 (2-5) 



4.3 
4.4 
4.6 



(3-5) 
(3-5) 
(4-5) 



4.6 (4-5) 



4,4 
4,4 

4,5 
4,5 

4,5 

4,5 
4,5 
4,4 

4,5 



mean range 
4.4 (3-5) 
4.6 (3-5) 

4.4 (3-5) 
4.4 (3-5) 

4.3 (2-5) 



4.5 
4.5 
4.4 



(3-5) 
(3-5) 
(3-5) 



4.6 (4-5) 
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Cost of Tr oininq 

• 

Total training cost for Site 2 subjects was $1,885.49, $62.15 a piece for June 
Workshop participants who hod slightly longer training lime, end $59.21 for eoch July 
Workshop subject. This total cost includes teacher and trainer costs for the time spent 
attending training sessions ($1,552.04 and $1 16.76 respectively) and cost of materials 
given to subjects during the sessions ($216.69). 

Follow-up After Training 

Twenty-three subjects requested additional information following training in the 
experimental procedures. Fifteen teachers requested information In all areas except 
on "drawing performance patterns". The most frequently requested area of 
Information was selecting a data type (eight teachers) and using decision-rules (six 
teachers). One teacher also requested information on compliance and eliminating 
inappropriate classroom behaviors. 

Four therapists requested oddltional information. Three therapists requested 
information on charting performance data. One therapist requested information on 
how to collect data, chart data, and select Instructional procedures for students with 
communication disorders in regular education. 

Six of the 23 subjects met Individually with one member of the 1H research staff 
following the presentations to provide the requested information. These meetings 
lasted approximately 10 minutes each, and were held within one week of th<* 
conclusion of training. Eighteen subjects requested Information to be provided in the 
fall when the school year begon. All 18 were contacted by phone, although some were 
provided help after they had actually begun to use the procedures* The follow-up 
conversations lasted approximately five minutes with the teachers (range 1.5 - 10 
minutes) and four minutes with the therapists (range L5 -10 minutes), and 10 minutes 
with each administrator. 

Cost of First Follow-up 

The total cost was $6!. 30 of which $36.1 2 was for the visits, and $25.18 for the 
phone conversations. 

Evaluation o f Handbook 

The thirty-one subjects who completed training were asked to rate the clarity of 
information presented in the Handbook of Experimental Procedures. The Handbook 
Questionnaire was completed By a total of lA subjects, including 10 teachers, three 
therapists and one administrator. The overall return rate for this questionnaire was 
45%t higher for teachers ond administrators (50%) than for therapists (33%). 

The fourteen subjects spent a median of 2 hours (range .25 - 8 hours) reviewing 
the Handbook of Experimental Procedures . Teachers generally spent more time 
reviewing the Handbo ok than therapists, which may be because teachers felt that th* 
experimental procedures were more applicable to their educational setting than the 
therapists (from Presentation Response Questionnaire). The one administrator spent 
five hours reviewing the Handbook . 

Teochers, therapists and the administrator all rated the information In the 
Hondbook of Experimental Procedure s as 3, 4, or 5 (Table 2-4). The sections on 
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drawing Hnes-of-pr ogress, determining percent corect, performance variability and 
sharp deceleration In corrects were rated slightly less clear (mean 4*1) them general 
considerations for instructional formats and information on acquisition (mean 4.6). 

Six teachers* three therapists and the administrator wrote additional comments 
concerning the Handbook, These subjects msst frequently wrote that they considered 
♦he information very helpful, practical end that the format was good* Two teachers 
expressed some concern about the amount of new information they hod to deal with 
and apply to their educational settings. There was one request for a Table of Contents 
or an Index* 



• 
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Table 2-4 

RATINGS OF CLARITY OF INFORMATION IN HANDBOOK OF EXPERIMENTAL PROCEDURES 



Content 



Teacher Rating 
NxtO 



Therapist Rating 

N=3 



Admnistrator Rating 
N=t 



All Subjects 
N=!4 





mean 


range 






mean ronge 


Using Decision-Rules 


4.4 


3-5 (N=9) 


4A5 


4 


4.4 


3-5 <N= 


Rules & Quick Reference 
Guide for Rate Data 


4.4 


3-5 


3,4,5 


4 


4.3 


3-5 


Rules & Quick Reference 
Guides for Percent 
Correct Data 


4.4 


3-5 


3,4,5 


4 


4.3 


3-5 


Quick Reference Guides 
for Change Strategies 




3-5 


1 f. r 

3,4,5 


4 


4.4 


3-5 


Drawing Llnes-of- 
Pr ogress. & Determining 
Percent Corect, Per- 
formance Variability, & 
Sharp Deceleration in 
Corrects 


it i 
4. 1 


3-D 


1 h t 

3,4,5 


4 


4.1 


3-5 


General Considerations 
for Instructional Formats 


4.6 


3-5 


4,4,5 




4.6 


3-5 


Information on 
Acquisition 


4.5 


4-5 


4,5,5 




4.6 


4-5 


Information on Fluency- 
Building 


4.3 


3-5 


3,4,5 




4.3 


3-5 


^ Information on Compliance 


4.4 


3-5 


4,5,5 




4.5 


3-5 


ERJC Medlon Rating 


4.5 




4.3 




4.5 
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Adoption of Procedures 

* 

Thirty-one subjects were moiled o Trial Period i Questionnaire, which asked 
subjects whether they planned to implement the experimental procedures in their 
educational settings. Eighteen subjects returned the Trial Period Questionnaire (return 
rate: 58%). Eleven subjects (61%), agreed to a trial period for the procedures in their 
classroom ("yes" subjects). These eight teachers and three therapists served a total of 
169 pupils. Seven subjects 08%) did not agree ("no" subjects); four teachers, two 
therapists and an odminstrator. 

Subjects who decided >ot to try the experimental procedures each gave a variety 
of reasons for their decision. Subjects visually stated that they did not have enough 
time to odopt the procedures, although one teacher reported insufficient time to 
report to the research project. One therapist reported that the procedures were 
incompatible with his/her curriculum and instructional procedures. One teacher 
decided not to odopt the procedures because hft/she didn*t know how to apply 
procedures to his/her educational situation, and another teacher said he/she could not 
see the need to use the procedures* These four subjects had previously rated the 
experimental procedures as applicable (4) following the training presentations. Other 
comments included personal reasons, and moving to a new district. 

The "yes" and 'Nio" subjects were compared on a number of variables. There were 
almost no differences between tne two groups in terms of the amount of adminstratlve 
and peer support in the subjects' educational setting. While both groups served the 
some types of handicapped pupils, the 'Yes" subjects had larger classes (8.5 pupils vs 
7.7 pupils) and more assistance (46 hours per week vs 40 hours), spent more time 
planning (3.7 hours vs. 3 hours) and were more likely to graph their data. 

Some differences in instructional practices existed between the two groups of 
subjects. "Yes" subjects collected data on a greater percentage of instructional 
programs (91% "yes" subjects vs 28% W subjects collect data on at least 85% of 
programs) and ran programs more frequently (64% vs 43% ran programs uaity) than the 
"no" subjects. "Yes" subjects also ran more trials in on instructional program than "no" 
subjects (64% vs 14% run 10 or more trials). 

The "yes** and "ho*' subjects were also compared on their responses on the 
Presentation Response Questionnaire and on the Handbook Questionnaire. "Yes" 
subjects rated their generc 1 attitude, and the usefulness and overall applicability of 
the experimental procedures higher than the •Yio" subjects. Mean ratings for the "yes" 
and Hno" subjects were 4.6 vs 4.2 on general attitude, 4.7 vs 4.6 on usefulness and 4.6 
vs 4. 1 on overall applicability. 

White the average "yes" subject spent more time than the average W subject 
reviewing the Handbook (3 hours vs 2 hours), the "yes" subjects consistently rated the 
Hmdbook lower on all sections than the Ino subjects. The median ratings for the "no" 
subjects were 4 8 (range 4.3-5.0) compared to 4.2 (range 3,8-4-5) for the subjects who 
agreed too trial period. 

Of the eleven subjects who ogreed to try the procedures, five either did not 
return questionnaires, or reported that they did not use the procedures. The six 
subjects who used the procedures included five teachers and one therapist. The six 
subjects served a total of 71 pupils per year (Table 2-5). 
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Table 2-5 

PUPILS SERVED BY SUBJECTS USING PROCEDURES 



Subject 


No. Pupils 


Age Range 


Handicapping Condition 


5207 


7 


6-11 


severely handicapped 


5IC! 


5 


2-3 


severely handicapped 


5108 


8 


16-21 


moderately, severely and profoundly handicapped 


5112 


? 


4-6 


mildly and moderately handicapped 


5IU 


7 


3-5 


severely handicapped 


6171 


35 


5-19 


moderately, severe!/ and profoundly handicapped 



0 
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The five teachers oil hod some classroom assistance, ranging from 76 hours F^r 
week to 30 hours per week. The therapist did not have assistance. 

Additional Follow-up 

Ten follow-up contocts were mode to four subjects using the procedures: one 
visit, eight phone colls end one letter. Follow-up infornrtation included compliance (3 
subjects), information on the use of the Decision Record Sheets (3 subjects), 
clarification of minimum federation lines (2 subjects) and information on selecting 
data types, setting aims, use of latency data, and designing initial instructional 
formats. 

Cost of Additional Follow-up 

The foltow-up costs during the trial period we#e $82^8. These costs include 
$10.87 for the visit, $4.71 for the letter, and $67.10 for the phone calls (see Table 2-13 
for per subject costs). 

Application of Procedures 

The five teachers appliea .'he procedures with 30 pupils in 95 programs for a 
total of i (8 weeks (42% of the total pupils served by these teochers). Two teachers 
reported once, one teacher reported twice, one reported three times and one teacher 
reported on four separate Procedure > Questionnaires. The therapist used the 
procedures far 18 weeks ?n 3 programs with 2 pupits (6% of the pupils served by this 
subject). The therapist reported once. 

The five teachers had indicatea prior to application of the rules that they 
preferred to collect accuracy data, and the therapist generally used counts of 
behavior. During the application of procedures, four teachers and the therapist 
collected accuracy data, although two of these teachers also collected some rate and 
duration data. One teacher collected rate data. Of the 95 programs, percent correct 
data were collected in 76 cases (80%), rate data in 9 cases (9%) and duration data in 10 
cases (11%). One teocher and the therapist used the rules ond procedures taught 
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during the workshops: drawing patterns for percent correct data. Two teachers used 
minimum ^celeratian lines and applied rate rules to percent correct data. One teacher 
tried both the minimum ■celeration and draw patterns rules for percent correct data. 
One teocher used the draw pjt terns procedures with rate data as taught at the 
workshops. 

The usefulness of the Handbook to subjects during the application of the 
procedures was given a mean overoge rating of 4.5, identical to the mean average 
rating of these six subjects prior to the use of the Handbook. As before, the technical 
sections (e.g., determining percent ige correct, drawing lines of progress, determining 
sharp variability) were ranked lower than sections on general instructional procedures 
(Table 2-6) except for Fluency-Building; possibly since few subjects moved pupils into 
fluency-building program procedures. Changes in ratings ore *hown In Figure 2-1. 
Despite the lower ratings of the technical sections, two of the three subjects who 
shared pupil performance data used the rules correctly. 

Table 2-6 

USEFULNESS OF THE HANDBOOK 

DURING APPLICATION OF THE PROCEDURES 

N»6 

Topic Rating 

Mean Range 

4.33 3-5 

4.83 2-5 

3.5 2-5 

3.6 1-5 
2.8 1-5 
4.0 2-5 
4.8 4-5 
4.67 3-5 
4.0 3-5 
4.83 " 4-5 
4.5 



• 


1. 


How to make on Instructional Decision 




2. 


Quick Reference Guides for Change Strategies 




3. 


Drawing Line s-of-Pr ogress 




4. 


Determining Percent Correct 




5. 


Determining Performance Variability 




6. 


Determining Sharp Deceleration in Corrects 




7. 


General Consideration for Instructional Formats 




8. 


Acquisition 




9. 


Fluency -Building 




10. 


Compliance 



1 1. Median Rating 
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FIGURE 2-1 

HANDBOOK RATINGS BEFORE AND DURING APPLICATION OF DECISION RULES 
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I 
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1 



SUBJECT 6'71 



W VERY CLEAR* 



"very confusing" 
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Prior to the use of the procedures, subjects reported planning for o median of 3*5 hours 
per week (range U5); during the first application af procedures* the median was 3 
hours per week (range 1-5)* One teocher spent more time, one teacher spent the same 
amount of time, and four others spent less time planning during the period tfwy were 
applying the rules (Figure 2-2)* 



Five of the six subjects rated their overall attitude toward the procedures as 5 
prior to app licet ion: of these five, three eventually ranked their attitude lower during 
actual application (Figure 2-3)* The subject who had ranked the procedures as 4*0 
prior to application ranked his/her attitude as 5*0 fallowing application* The therapist 
gave the lowest overall rating to the procedures, and ranked charting and drawing 
I ines-of -progress as 3 end I respectively* 
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Impact on Pupil Performan ce 

Four subjects estimated that the procedures accelerated pupil performance, 
while one subject estimated that pupil progress decelerated, and one estimated that it 
accelerated in some programs and decelerated progress in other programs* Both of 
these latter two subjects applied the rules to percent correct data* All of the subjects 
plan on continuing to apply the procedures* 

Three subjects shared pupil performance data on a total of 19 pupils in 52 
programs, including a wide variety of target behaviors (Table 2-7)* 

Subject 5101 submitted percent correct data on many programs* According to 
this subject, the experimental decision rules were applied in all programs* However, 
no evidence of the application of rules was found in the data of nine programs (Table 
2-8)* Rules for rate data applied with the minimum 'celeration procedure v*ere used in 
eleven programs, and trie draw patterns procedure and the percent correct rules were 
used in 4 programs* The charted data indicate that Subject 5101 may not have applied 
the rules correctly: programs were not changed when the pupil met performance 
criterion; changes were not made either when performance patterns were decelerating 
(draw patterns procedure) or when performance fell below the minimum 'celeration 
tine, and patterns and minimum "celeration lines were not drawn for many of the 
phases* 

The four programs in which the draw patterns procedure was used covered a 
total of 93 days of data (Table 2-9) and there were a total of seven patterns drawn, 
when 13 could have been drawn using seven day lines-of-progress, or 18 using five day 
Itnes-of-progress* In II programs Subject 5101 applied rules using the minimum 
'celeration procedure* Interventions were generally made late, when the pupil was 
more than three days below the line, or not at all (six programs had no intervention 
changes). Of the 9 usable interventions, all accelerated pupil progress, and the rules 
were occurate in 60% of the cases which could be anal/zed (Table 2-10)* This subject 
estimated that the rules accelerated pupil performance* A comparison of programs in 
which there is no evidence of rule application and those in which the rules were 
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FIGURE 2-2 



HOURS PLACING BEFORE AND DURIN6 APPLICATION OF DECISION RULES 



BEFORE DURING 

5 



CO 



1 



SUBJECT 5101 



BEFORE ' DURING 

5 mx ; x 



CO 



I 



2 I I 



SUBJECT 5108 



CO 



BEFORE! DURING 

5 r x 

- I x 

I 



1 [ ~ 



SUBJECT 5112 



- X 



CO 



1 



I 



SUBJECT 5207 



1 



X J X 
SUBJECT 5116 



5 _ 



CO 



- I 
I 



-X I X 



1 



SUBJECT 6171 



ERIC 202 



203 



FIGURE 2-3 

ATTITUDE TOWARD PROCEDURES BEFORE AND DURING APPLICATION OF DECISION RULES 
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opplied with minimum toleration procedures Indicotes that the rules may hove 
increased the number of interventions made by Subject 5101. However, since the 
target behaviors are different, tt ts impossible to draw any conclusion;;. 

Subject 5108 applied the rules for rate using the minimum toleration procedure 
to percent correct data, and generally applied the rules correctly: stepping ahead 
when the student was at aim 27 or 27 times, and changing instructional strategies 12 
of the 16 times pupil performance fell below acceptable progress. In those four 
instances, the subject drew new minimum toleration lines rather than change 
instructional procedures. Eleven of the sixteen interventions mode in 25 programs 
were successful (69%) (Table 2-9). Subject 5108 utilized the suggested strategy 69% of 
the time, and tht rules were 75% accurate in predicting the outcomes of decisions 
(Table 2-10). The ?5 programs included five programs aver a total of 351 calenaar 
days and 155 days of data in which no changes of any kind were made. This subject 
estimated that the use of the procedures accelerated pupil performance. 

SiAject 5207 also applied the minimum toleration proced'Jre and rules for rate 
data ta percent correct data, and generally applied the rules correctly: stepping ahead 
when the student was at aim 17 of 20 times, and changing instructional strategies nine 
of 1 1 iimes pupil performance fell below the minimum toleration line. Subject 5207 
applied the rules in 91% of the opportunities, 91% of the interventions were 
successful, and the rules predicted the impact of interventions 82% of the tittie (Tables 
2*9 and 2-10). In one case, the subject did not follow the rules and pupil performance 
improved, (n another case, the subject fallowed the strategy change, but had drawn 
the minimum toleration line incorrectly and made o change when one was not 
required. The change failed ta improve pupil performance. 

As a group, the three subjects who shared pupil performance data collected 481 
days of data in 48 programs, and made 92 changes. Of the 72 changes, 54 were 
changes mad* to move the pupil forwad in the curriculum and 38 were strategy 
changes. Eighty-four percent of the changes were made in accord with the rules, 83% 
of the changes were successful, ond the rules accurately prediced the outcomes of the 
changes in 77% of the cases. 

Accuracy of the Rules 

Notations made by the subjects of the types of strategy changes they mode were 
used ta determine whether or not an intervention was made in accord with the 
strategies included in the rules. Thirty-one of the 35 usable interventions included 
clear descriptions of strategies; the other four changes were eliminated from this 
onotyes. The predictability of the rules for each subject varied: 60% for 5101, 75% 
far Subject 5 1 08 and 82% far Subject 5207. The accuracy of the rules in each case are 
shown in Table 2*1 1. 

Cast Analysts 

The total costs for training 31 subjects in this site are shown in Table 2-12. Of 
the 31 trained subjects, six octuolly utilized the procedures in programs for thirty 
pupils over a total of 18 weeks. If ali site costs are included, these figures result in an 
average per pupil cast of $65.47 during the experimental period of application, or 
approximately §17.20 per week as the costs of applying the experimental procedures. 

If costs are separated ta include only those costs incurred in trc:ning subjects 
who actually then used the procedures, these figures are reduced to a median cost of 
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$8.49 per pupil and $4.08 per week (Table 2*13). For subjects who applied the 
procedures, the cost per prcgram per week averaged $.23. Figures for each subject 
vary considerably (Table 2-13), spending an the numbers of pupils and programs and 
length of time the subject applied the procedures. 
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Table 2-7 

TARGETED MOVEMENT CYCLED IN PUP... PERFORMANCE OATA 



Age Range of Pupils Movement Cycles 
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1-4 grasps toy; pulls toy by string; rolls ball; plays "push to go 1 *; puts object In container; cruises; 

walks in walker; play* musical instrument; takes sock off; strings beads; holds cup; Imitates 
play; rides tricycle; puts object in foam template; touches ball; points to named object; 
props self up cn elbows; stands; responds to auditory stimuli; responds to tactile stimuli; 
points to named picture; opens hands; turns head to touch. 

6-10 matches object to picture; matches colors; tc'-e and put pens In jar; tdke shirt off; matches 

objects; puts pants on; respona* to name; scoops food. 

16-21 collates sheets; give requested no* pennies; puts paper in folder; assembles sifter; sorts 

objects; puts shirt on; reoches for noise maker; plays with toys; says 'beside box" tup to 
cups" '*shoe to shoes**; counts out 4 pieces. 
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Table 2-B 

>tVIDUAL PUPIL PROGRESS ON PROGRAMS BEFORE* AND DURING APPLICATION Of EXPERIMENTAL PROCEDURES 

SUBJECT 5101 









Before Uie of t 


he Roles* 








During Use of Roles 




Pn>ii 


Behovlor 


Ooto 
Doys 


Step Av. Days 
Aheadi To Step 
Aheod 


Inter* 
vent ions 


% Success 
hrtefvMV 
tlons 


Ooto 
Days 


Step 
Ahead* 


Av*Doys 
To Step 
Aheod 


Inter* % Success 
vent* Intervene 
Ions ions 


% Follow 
Rules 


i 


{prop toy 


29 


0 ? 


2 














I 


pot obj+ in 










52 


1 


52 


4 100% 


7 


I 


opproprlare 


23 


2 12 


0 


7 


2D 


1 


20 


1 100% 


0% 


2 


cruise 


3 


met aim 
















2 


wollt in 
wollcer 


7 


rn*t aim 
















2 


ploys rhythm 
Instrument 










21 


0 


*> 


0 ? 


7 


2 


sock off 










S3 


2 


26 


J 100% 


100% 


3 


string beads 


7 


met Qtm 
















3 


oVinks 


22 


1 22 


0 


? 












• 

5 


taltote ploy 
ride trilte 


\e 

27 


1 16 
1 27 


0 
0 


? 












5 


put in 










35 


1 


35 


2 ? 


7 


5 


string beods 










40 


3 


13 


0 ? 


7 


s 


touch boll 










25 


0 




0 ? 


7 


6 


props on 
elbows 


28 


0 ? 


0 


7 












6 


■■prone boord* 










35 


1 


35 


0 ? 


7 


6 


respond to 

tactile 

stimulation 










31 


1 


31 


0 ? 


7 


6 


respond to 

auditory 

sfimulofion 










46 


0 


46 


1 too% 


0% 


7 


turn head 










55 


0 


7 


2 100% 


100% 



♦Programs In which no evidence <A vne application of the rules appears in pupiF performance ooto 
Summary 



media* data doys ■ 25 median data doys = 35 

prog, w/changes (S*A* or Inter) a 5 of 6 prog w/chonges - 9 of 1 1 

prog. w/$.A. ■ 4 of 5 prog* w/S*A» ■ 7 of 9 

median doys to step ahead * 16 median doys to step ahead = 3t 



prog, winter * I of 5» 0% tuccwful ^ prog* w/irtter * 6 of 9 

20% of changes were Interventions ~* w% of changes were interventions 



Table 2-9 



ANALYSIS OF PUPIL PERFORMANCE DATA 
Application of the Prow Patterns Procedure 



Calendar 

Subject Pupils Programs Days 



Data 0 No Patterns # No Chonge 
Days Drawn Patterns 



# Change Proce- 
dure Patterns 



Inter- # Success* % Success* 
vent ions ful Decision fut 



sioi a a 276 93 7 a 3 13 43% 

•Stfjject estimated that the procedures accelerated pupil performance, 57% utilization of suggested changes, predictability of rules, was 57%. 



Subject Pupils Programs 



Application of the Minimum KTelerotion Procedures 



Calendar 
Days 



Data 

Days 



Phose* 



Step 
Aheads 



Inter- 
ventions 



Usable 



fSuccess- %Success- Subjects Estimate < 
ful ful Impac* of Procedure 



5101 


6 


II 


l"2l 


413 


30 


7 


II 


9 


9 


100% 


5108 


7 


25 


177^1 


838 


69 


27 


16 


16 


II 


69% 


5207 


6 


II 


987 


310 


41 


20 


II 


II 


10 


91% 


TOTAL 


19 


48 


3782 


481 


140 


54 


38 


35 


30 


86% 



Accelerated proy< 



BEST COBf AVAILABLE 
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Table 2-10 

UTILIZATION OF SUGGESTED STRATEGIES 

Subject 9 Interventions 0 Usable # Successful % Successful # Using Suggested % Using Suggested Predictability 

Strogegy Strategy of the Rules 

5101 II 9 9 100% 3ofS» 60% £0% 

5108 16 16 II 69% H 69% 75% 

5207 II II 10 91% 10 91% 62% 

TOTAL 38 36 30 83% 2* 8*% 77% 

*■ 

•Phase notes for fcur phases were Incomplete or illegible and so these phases were excluded from analysis* 
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Table 2-11 



ACCURACY OF RULES FOR INTERVENTION CHANGES 



Pupil Progress Pupil Progress 

Actuolly Accelerated Actually bid Not Accelerate 

Changes mode in occord with a rule 64% 9% 

(Prediction that progress will be 

occeleroted) (N-2 1 ) (N=3) 



Changes mode not in occord with a 15% 9%* 

rule (Prediction that progress will 

not be accelerated) (N=5) (N=3) 



♦Concurrence of prediction and actual results indicates rule accuracy: total 24/32=75%. 

Table 2-12 
COST SUMMARY 



ALL COSTS 

No. Subjects Trained : 31 

Total Training Costs : $1885.49 

Total Follow-up Cost : $ 143.98 

TOTAL SITE COST : $2029.47 

Av. Per Subject Cost : $ 65.47 

Av. Per Pupil Cost* : S 67.64 

Av. Per Week Cost** : $ 17.20 

* Includes N=30 pupils whose teachers applied +he procedures in instructional programs 
** Includes N*l 18 weeks during which teachers applied procedures 




Table 2-13 

COSTS FOR SUBJECTS APPLYING PROCEDURES DURING EXPERIMENTAL PERIOD 
Appllcotion of procedures All Cost? Cost Analysis 

Per 



Subject 


Weeks 


Pupils 


Programs 


Training 


Fol low-op 


Total 


Per Pupil 


Per Program 


Per Week 


Program/Week 


5101 


28 


9 


40 


$ 


62.15 




14.28 


$ 76.43 


$ 8.49 


$ 


1.91 


$ 


2.73 


$ 


.07 


5108 


II 


8 


25 


$ 


62.15 


$ 


2.16 


$ 64.31 


$ 8.04 


$ 


2.57 


$ 


5.85 


$ 


.23 


5112 


13 


3 


3 


$ 


62.15 


$ 


1.65 


$ 63.80 


$ 21.27 


$ 


4.91 


$ 


1.64 






5116 


17 


1 


2 


$ 


62.15 


$ 


7.27 


$ 69.42 


$ 34.71 


$ 


4.08 


$ 


2.04 






5207 


31 


7 


16 


$ 


59.21 


$ 


78.34 


$ 137.55 


$ 19.65 


$ 


8.60 


$ 


4.44 


$ 


.27 

*■ 


6171 


18 


2 


3 


$ 


59.21 


$ 


0.00 


$ 59.21 


$ 29.60 


S 


9.87 


$ 


3.29 


* 

$ 


.55 


TOTAL 


118 


30 


95 


$ 376.08 


$ 103.71 


$ 470.72 
















MEDIANS 
















$ 66.87 


$ 8.4? 


$ 


2.57 


$ 


4.18 


$ 


.23 
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SITE 3 
Introduction 

The participation of the teachers and therapists working with severely 
handicapped pupils in the Olympia, Washington, school district was discussed with 
Or- Stitlman Wood, Director of Special Services for Olympia. Dr. Wood, who has long 
advocated the use of Precision Teaching in the district, Indicated that special 
educators and their advisors in the district were familiar with the research project and 
wanted to receive inservice training on the use of decision-rules. An orrangement 
between the Olympia School District and this project was made: project staff would 
train the advisors of alt special educators in the use of the decision-rules ond the 
Handbook of Experimental Procedures, white teachers and therapists of 
"development ally disabled 11 pupils (Including multiply, severely, profoundly and 
moderately handicapped pupils) would be invited to participate as subjects. 

A four hour training session with the advisors was conducted by Dr. White, Dr. 
Liberty, and Ms. Martin on August 28, 1979. Topics discussed during the session 
included a history of the development of data decision-rules, the use of decision-rules 
with minimum federation, decision-rules for use with rate and percent correct data, 
and different types of Intervention strategies. Advisors were given an opportunity to 
practice drawing tines-of "progress and received a handout on different data types. A 
copy of the Handbook of Experimental Procedures was distributed to each of the six 
advisors who attended. At the conclusion of the meeting it was decided that the 
advisor far the teachers serving developmental ty disabled pupils would arrange a 
meeting with those teachers prior to the start of the school year to discuss their 
participation in the project. 

A 45 minute meeting with teachers and therapists serving pupils labeled 
♦Uevelopmentally disabled" was held on October 10, 1979 by Dr. White and Dr. Liberty. 
Following a general introduction by Dr. Stitlman Wood, Dr. White presented a history 
of the decision-rules and Dr. Liberty gave a short overview of the project. Eight 
potential subjects were given copies of the consent form (Appendix 3-1) and 
Background Questionnaire and asked to complete and return the forms by mail if they 
wished to participate In the project. Additional forms for potential subjects who were 
unable to attend the meeting were distributed by their advisor. A total of 13 subjects 
eventually consented to participate in this year's research. 

In comparison with other sites, the training received by subjects at Site 3 was 
generally the least extensive, except for Site U The majority of subjects were troined 
in two large group sessions, three subjects were trained in smalt group sessions^ and 
make-up sessions were scheduled for those who couldn't make the regular sessions. 
Subjects were also prc/lded follow-up assistance, by project staff during their 
participation in the study. 

Method 

Subjects 

The 13 subjects included nine teachers, two communication disorders speciclists, 
and one occupational and one physical therapist. One of the seven teachers had no 



' Two consenting teachers did not return Background Questionnaires or other 
ERXC questionnaire and are not Included in demographic Information provided in this section. 
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previous experience, while the six experienced teachers averaged 6.7 years teaching 
(range l-!3 years) and 4.3 years teaching special education (ronge (-10 years). Six of 
the seven teachers hod prior training in Precision Teaching. The four therapists 
averaged 4.5 years (ronge 1-8 years) in special education. Two of the four therapists 
had prior training in Precision Teaching. 

The seven teachers served a total of 63 pupils per year and 5? on on average 
school day. The mean class size was 8*4 pupils with a range of 6-12 pupils. Most 
teachers served a mixture of pupils labeled either moderately, severely or profoundly 
handicapped (Table 3-1). The four therapists served a total of 124 pupils per year, and 
a total of 40 on an average day. Client load was 7, 6, 10 and 17. The age ranges of 
the pupils served ore shown in Table 3-2, with most of the classrooms serving 
preschool or elementary school pupils. 



Toble 3-1 

PUPILS SERVED E>* CONSENTING SUBJECTS: SITE 3 



Learning Mildly Mode rote! y Severely Profoundly Other* Total 
Subjects Disabled Handicapped Handicapped I- ttdicapped Handicapped Pupils 



Teachers 
N=7 



^^ei 



N=4 



rapists 



7 
17 



23 
25 



18 
44 



14 



10 



10 



62 



10? 



Total 3 24 48 62 24 10 171 

•Communication disorders 

Toble 3-2 



AGES AND APROXIMAfE SCHOOL LEVELS FOR PUPILS SERVED BY SUBJECTS: SITE 3 



No. Pupils 


Ages (in years) 


School Level 


Subjects 


27 


2-6 


Preschool 


2 teachers 


23 


6-12 


Elementary 


3 teachers 


13 


12-22 


High School 


2 teachers 


75 


t-12 


mixed 


2 therapists 


4? 


1- 18 


mixed 


2 therapists 
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As e group, the teochers hod more help available In their classrooms than did the 
therapists* Each of the seven teachers reported some assistance. There was a total of 
M paid assistants f four volunteers, four student teachers and eight other people 
assisting teachers for a total of 369 hours per week* The subject with the most help 
had 10 people helping for 122 hours per week, while the subject who reported the least 
help had one person assisting for 29 hours per week. In general, teachers had 
approximate)/ four persons providing some type of assistance for an average of 53 
hours per week. One of the four therapists hod the help of one paid assistant for six 
hours per week; the other therapists did not have assistance. Pupils in six of seven 
classrooms averaged 7.8 hours of therapy per week (range 3 to 10 hours). The teacher 
without therapy service taught one moderately handicapped pupil, two severely 
handicapped pupils and three profoundly handicapped pupils. 

The seven teachers averaged more planning time (mean = 4.4 hours, range 2-10 
hours) than the therapists (mean * 2.8 hours, range 2*5 hours). Six teochers provided 
o least three different types of Instruction— requesting individual responses in a l;l 
setting (mean 55% of the time, range 25-80%), requesting pupil responses at natural 
occasions (mean 46% of time, range 10-100%), and requesting pupil responses in group 
situations (mean 21% of time, range 5-50%). One subject spent 80% of the 
instructional time in a l;l setting and 20% of the time using natural occasions. All 
four therapists spent at least 90% of their time requesting individual pupil responses 
during 1:1 instruction. 

As a group, the teachers ran programs much more frequently thon the therapists. 
Five of the seven teachers ran programs at least one present session per day and two 
teachers ran programs four times each week. One therapist saw each pupil three 
times per week; two, twice a week; and one therapist conducted programs with each 
pupil once a week. 

Two teachers and two therapists generally provided fewer thon 10 response 
opportunities, three teachers and one therapist provided between ten and 20, and two 
teachers and one therapist more thon 20. Four teochers collected data on 95*100% of 
their programs, two teachers on 85-94% and one teacher on 75-84%. Four teachers 
collected data every time they conducted an instructional program* and three teachers 
collected data just about everytime. Teachers differed in the number of trials on 
which *hey collected data during their data-based programs. Three teachers reported 
that data were collected on every trial. Four teachers collected data on either the 
first or last trial-only* Although teochers reported using all of the different data types 
ii their !issrooms, five of the seven teachers collected rate data most often. 

Two of the four therapists collected data on 95-100% of their instructional 
programs, while the other two collected data on 50-74% wj their programs. One 
therapist collected data every time the program was run; while the other three 
collected data about half the time. One therapist collected data on every trial during 
the program i one therapist collected data on special probe trials, one therapist on 
three to ten trials per program and one on a variable number of trials depending on the 
program. None of the therapists used rate data in their programs. Two therapists 
reported that behavioral counts were the most frequently used data types. The other 
two therapists reported using percent correct data and time-based data for most of 
their programs. 

All of the subjects graph their data, and all hod at least some rules for making 
decisions. All hod rules for moving the pupil to the next step and for when the pupil 
met the aim for at lease some programs. Eight of 1 1 did not have rules for deciding 
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how to modify instructional procedures and seven did not have rules for when to 
change the instructional strategies* : 

Training 

The first general training session was attended by seven subjects, two advisors, 
and three af the subjects' Instructional aides* Or. Liberty explained details af 
participation in the project and discussed prerequisites for use af the decision-rules 
and different data types* Collection and charting procedures for regular rate and 
adjusted rate data were discussed and demonstrated by project staff. Session length 
was approximately one hour, and handouts to subjects attending this session included 
••Prerequisites on Using the Deciston-ftules" and a "Seconds ta Decimals" conversion 
sheet (see Appendix 3). 

Fallowing this session, the advisor af the subjects recommended dropping the 
explanation af data types, collection and charting procedures from training. 
Therefore, the four subjects who hod missed the first general training session and 
attended make-up pessions did not receive this information. These subjects attended a 
40 minute session. ' 

The second general training session was attended by ekjit subjects, three aides, 
and the advisor for these subjects. The use af the Handbook of Experimental 
Procedures and the decision-rules was emphasized for this one hour session. AH 
subjects received the Handbook at the beginning af this session. Subjects were given 
opportunities U practice drawing minimum 'celeratlon lines and lines-of-pragress 
fallowing demonstrations by project staff. Verbal explanations af the flow chart of 
the decision-rules in the Hcndbook were followed by a practice period in which 
subjects were asked to determine two types af instructional decisions by applying the 
rules to charted data. Project staff discussed the "Quick Reference Guide for 
Acquisition" and how ta use it. Procedures for drawing a phase change line and 
determining a new minimum 'celeration line were demonstrated. Concluding remarks 
on the requirements far further participation by subjects in this year's study were 
fallowed by distribution af the Presentation Response Questionnaire to all subjects. In 
addition ta the Handbook, subjects attending this session received practice sheets an 
drawing a I ine-of -progress, and two charts af data for practice in applying the rules. 
One subject was unable ta attend the second general training session and attended a 40 
minutes make-up session. 

In addition to the group sessions on using the Handbook and the decision-rules, 
subjects received foHaw-up assistance from project staff if they electc 1 ta try out the 
procedures in their classroom. 

Results 

Evaluation of Training 

Seven of nine teachers and all four therapists completed a Presentation Response 
Questionnaire at the conclusion of training. Two teachers completed training but did 
not return the questionnaire ta project staff. 



Two subjects were trained separately, but never completed any questionnaires or 
used the procedures. They are not included in this group. 
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A five point Llkert Scale was used to measure subjects* ratings of the clarity of 
Information presented during training* Tenchers* mean rating for the overall clarity of 
Information presented during training was 4.0 (range 3-5) or "clear" while the 
therapists* ratings were 5, 3, 3 and X In addition la the overall rating, subjects 
evaluated the Information presented on eight specific areas of training (Table 3*3). 

Teochers and therapists rated the clarity of some topics differently. Teachers 
gave a high rating to the Information presented on collecting data (mean 4.6, range 4- 
5) white the therapists rated this topic slightly lower In clarity (4,4,3,3). Therapists 
gave a lower rating for the information presented on establishing initial Instructional 
formats (4,3*3,3) than did the teachers (mean 4.4, range 4*5). However, therapists 
rated the presentation on using the Handbook hlgjher (5,5,4,4) than did the teachers 
(mean 4. 1 , range 3*5). 

Four teachers end two therapists wrote comments regarding training* Two 
teachers felt that more time should have been spent on setting aims* One teacher and 
one therapist felt that the training time was too short. Comments by two teachers 
and one therapist pertained to the usefulness of the decision-rules ond the 
xperi mental procedures In general, e.g., "My overall impression of the purpose and 
content of the instructional Hierarchies Research Project is very positive/ 
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Table 3-3 

SUBJECTS' RATINGS OF CLARITY OF INFORMATION PRESENTED DURING TRAINING 

Presentation Topic Teacher Rating Therapist Rating All Subjects 

<N*7) <N=4) <N=II) 





mean 


Vrarge) 




mean 


(range/ 


Genera! purpose of the procedures 


4.6 


(2-5) 


5,5,4,4 


4.6 


(2-5) 


Establishing initial Instructional formats 


4.4 


(4-5)* 


4»?»3»3 


3.9 


(3-5)** 


Collecting performance data 


4.6 


(4-5) 


4,4,3,3 


4.2 


(3-5) 


Charting performance data 


4.7 


(4-5) 


5,5,5,3 


4,6 


(3-5) 


Drawing patterns 


3.9 


(3-5) 


5,5,4,3 


4.0 


(3-5) 


Using decision-rules 


4.3 


(3-5) 


5,5,4,4 


4.4 


(3-5) 


Types of instructional changes 


4.0 


(3-5) 


4,4,4,3 


3.9 


(3-5) 


Using the Handbook 


4.1 


(3-5) 


5,5,4,4 


4.3 


(3-5) 


Overall 


4.0 


(3-5) 


5,3,3,3 


4.0 


(3-5) 



* N=5 subjects respond to this question 
** N=9 subjects respond to this question 
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Attitude Following Training 

Subjects rated their general attitude toward the experimental procedures* The 
teochers averaged 4.3 (range 4-5), or between "Favorable* and "Very Favorable"* 
Three therapists ratings were 5,4 and 3, and one therapist did not provide a rating* 

Subjects also rated how useful the experimental procedures might generally be to 
teochers* Subjects' average rating was 4.5 (range 4-5) or between "Very Useful" and 
"Useful*. 

Teochers rated the overall applicability of the procedures ta their own 
instructional situations as 4.4 (ran?- 4-5) or between "Very Applicable" or 
"Applicable". Therapists' ratings were 4, 5, 3, and 3 (Table 3-4). Teochers generally 
found the experimental procedures "Applicable" or "Very Applicable" to their 
instructional situations. Teochers rated the information on designing initial 
instructional formats and applying change strategies slightly less applicable (mean 4.4) 
than the other information presented (mean 4.4, range 4-5). Information on conducting 
Instructional formats, collecting and charting performance data were considered the 
most applicable (mean 4.7, range 4-5). 

The four therapists rated the experimental procedures as slightly less applicable 
to their own instructional situation than did the teochers, and showed a greater range 
in responding* The information on designing initial instructional formats was rated the 
least applicable by therapists (4, 4, 3, 2) while the information on ejecting data (5, 4, 
4, 3) and conducting instructional programs (4, 4, 4, 4) were rated the highest. In 
general, there was no clear relationship between subjects' ratings of the clarity of 
information presented during training ami their general attitude toward: the 
procedures* However, subjects rated the usefulness of the procedures ta teochers In 
general (mean 4.5, range 4-5) slightly higher than the applicability of the procedures to 
their own instructional situation (mean 4.0, range 3-5). 

Subjects' instructional practices end educational bockgraund were examined to 
determine If these variables might influence subjects' atttitudes towards the 
experimental procedures. Three patterns were evident. The three siAjects with no 
Precision Teaching background rated the usefulnes of the procedures In lower (4, 4, 4) 
than subjects with Precision Teoching (mean 4.6, ronge 4-5). The some relationship 
was found between their ratings for overall applicability of the procedures (4, 4 t 3 vs 
mean 4.3, range 3-5), general attitude (3, 4 vs mean 4.4, range 4*5) and Individual 
topics covered in training* 

Five of II subjects reported that rate data was the data type most frequently 
used In their classrooms. These subjejts rated their general attitude towards the 
procedures higher (X = 4.4, range 4-5) than did the other subjects (X « 4.0, range 4-5). 
A similar pattern was evident in ratings by the five subjects who conduct daily 
programs and the six subjects who conduct programs less frequently. The former 
group of subjects gave a much higher rating far their attitude towards *Se procedures 
than the latter group of subjects for all topics of training. 

In general, subjects with previous experience in Precisian Teaching and who 
apparently u^d systematic instruction In their educational settings rated their 
attitude and the applicability of the experimental procedures higher than subjects 
without this background. There was little, if ony, relationship between subjects' 
ratings af clarity of the topics and their attit»*de towards the procedures. Teochers 
rated the experimental procedures more applicable ta their instructional setting than 
did the therapists. 
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Table 3-4 

SUBJECTS' ATTITUDE TOWARD THE EXPERIMENTAL PROCEDURES 

Procedure Teachers Therapists All Subjects 

(N=7) (N=4) (N=ll) 





meon 


(range) 




meon 


(range) 


Overall attitude 


4.3 


(3-5) 


3,5,4 


4.2 




Usefulness of procedures 


4.4 


(3-5) 


4,4,5,5 


4.5 




Applicability 
















V*— Jf 


i u i 

J, 4, 3, J 






Designing instructional formats 


4.4 


(4-5) 


4,4,3,2 


4.0 


(2-5) 


Conducting Instructional programs 


4.7 


(4-5) 




4.1 


(3-5) 


Collecting performance data 


4.7 


(4-5) 


5AM 


5.4 


(3-5) 


Charting performance data 


4.7 


(4-5) 


5,4,3,3 


4.2 


(3-5) 


Using decision-rules 


4.6 


(4-5) 


4,4,3,3 


4.2 


(3-5) 


Applying change strategies 


4.4 


(4-5) 


5,4,3,3 


4.2 


(3-5) 
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Cost of Training 

The totol site cost was $386,65, Training costs for the eleven subjects trained os 
o group in two sessions wos $35.1 5 per subject. These costs include trainer time (2.69 
per subject), subject time (25-76 per subject) and Handbook cost* ($6.75 each). 

Follow-up After Training 

One teacher requested additional assistance on designing instructional formats, 
selecting a data type and selecting aims for oil day programs* Follow-up was provided 
in person by one trainer during a 20 minute session. Two therapists requested more 
information on selecting a data type and applying the changes to specific programs. 
One of these therapists also requested information on using the decision rules. For one 
subject, direct follow-up was provided by one trainer in a 20 minute session, while the 
other therapist discontinued participation in the study before follow-up could be 
provided. 

The total cost of the two follow^ visits was $14.34. 

Evaluation of Hondbook 

Subjects evaluated the clarity of information presented *n The Handbook of 
Experimental Procedures on the "Handbook Questionnaire. n Thirteen questionnaires 
were distributed aid six of seven teachers and two of four therapists returned 
completed questionnaires (73%). Teachers averaged two hours reviewing the Handbook 
(range 1.5-3 hours), one therapist spend 30 minutes and one therapist spent two hours. 

Subjects rated the clarity of information presented in eight different sections of 
the Handbook on a Ukert scale from 5 (%eiy clear'*) to I ("confusing). Subjects 
generally rated the information as either "4" or "5" (Table 3*5). Teachers and 
therapists differed in some of their ratings. Teachers rated the sections of the 
Hondbook, on drawing I ines-of -progress, determining percent correct, performance 
variability aid sharp deceleration in corrects the least clearly presented (mean 4.3, 
range 3*5), but generally give high ratings for other sections. The two therapists rated 
the decision-rules for use with rate data and the quick reference guides less clear ("3" 
and "4") than other sections of the Hondbook . 

Three teochers provided written comments. Two wrote that the overall content 
of the Hondbook was very good (e.g., "very clear and extremely well written'*)* One of 
these subjects, however, tndicoted that better organization of the materials was 
needed, due to the number of revisions and additions. The other teacher praised the 
section on "compliance", but reported that the section on "acquisition" was "hard to 
fallow." 
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Table 3- 

SUBJECTS* RATINGS OF CLARITY OF INFORMATION PRESENTED IN THE HANDBOOK 



Topic 



Teachers 
<N*6) 



Therapists 

(N=2) 



All Subjects 
<N=8) 





mean 


(ronge) 




mean (range) 


Using the Decision Rules 


u j 


(4-5) 


5,5 


4.8 


(4-5) 


Rules for Rote <nd the Quick Reference 
Guide for Rules for Rate 


4.6 


(4-5)* 


3,4 


4.3 


(3-5)»» 


Quick Reference Guide for Change Strotegies 


4,8 


(4-5) 


4,4 


4.6 


(4-5) 


Li nes-of -progress, determining percent 
correct, performance variability & 
sharp deceleration in corrects 


4.3 


(3-5) 


5,4 


4.4 


(3-5) 


Genero) consider ot ions in initial 
instructional formots 


4.7 


(4-5) 


4,4 


4.5 


(4-5) 


Information on Acquisition 


4.7 


(4-5) 


5,4 


4.6 


(4-5) 


Information on Fluency-Building 


4.8 


<4-5)» 


5,4 


4.7 


(4-5)»» 


Information on Compliance 


4.8 


(4-5) 


5,4 


4.8 


(4-5) 


Median Roting 


4.7 




5,4 


4.8 





* N=5 subjects responded to this question 
** bk7 subjects responded to this question 
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Adoption of Procedures 

All tl subjects returned o •'Trial Period Questionnaire" (100%). Six of the II 
(55%) agreed to do a trial period. All were teachers, serving a total of 47 pupils each 
year (Table 3-6). 

The five subjects who chose not to adopt the procedures ("No" subjects) 
explained their decisions. Three of the four therapists and two teachers reported that 
they did not have enough time to use the procedures. One therapist noted, in addition, 
that the procedures were incompatible with the curriculum. 

The "yes" ctkJ "no" subjects were compared on a number of variables. Both 
groups reported administrative support. However, five of six "yes" subjects reported 
"peer support** (other teachers in their school using the procedures) while only one of 
the seven "ho" subjects were aware of any other teacher or therapist using the 
procedures. "Yes" subjects averaged fewer pupils per day thon "no" subjects (mean 7.8 
versus mean 10.4), more planning time (mean 4.8 hours vs. mean 2*6 hours) and more 
assistance in the classroom. Five of the six 'Yes** subjects reported previous training 
in Precision Teaching, while only three of the five "ho" subfects had had such training. 
Some differences in evaluation procedures were found between the two groups of 
subjects* In general, •yes" subjects collect data on a higher percentoge of their 
instructional programs (4 of 6 "yes" subjects collect data on 95-100% of their programs 
versus 2 of 5 "no" subjects), on a more consistent basis (all six "yes" subjects collect 
data almost every trial vs. 2 of S "ho" subjects) and use rate data more often (5 of 6 
"yes" report rate dato as the most frequently used data type versus I of 5 "ho" 
subjects). There was no difference between the "yes" ctkJ "no" subjects in their rat?ng 
of the Handbook or the training. They did differ, however, in their attitude toward the 
experimental procedures following training) "yes" subjects roted the procedures as 
more applicable than **no" subjects (mean 4.3 versus mean 3.8). Both groups gave 
similar rating in other areas, including the "usefulness" of the procedures. 

Table 3-6 

PUPILS SERVED BY SUBJECTS USING PROCEDURES 



Subject 


No- Pupils 


Age Range 


Handicapping Conditions 


5402 


15 


1-5 


mild, moderate f severe, profound 


5403 


7 


5-10 


moderate f severe, profound 


5404 


5 


8-11 


moderate, severe, profound 


5405 


6 


6-11 


moderate, severe, profound 


5413 


7 


13-17 


moderate, severe, profound 


5414 


7 


14-17 


severe, profound 



Additlonol Follow-up 

Twenty-five follow-up contacts were mode to six subjects during their 
application of procedures! 13 visits ond 12 phone conversations. Topics covered 
during the visits included: review of minimum *celerotion procedures, setting aims, 
selecting program targets and establishing initial instructional formats, fluerK/ 
building strategies cjkJ aim setting, data c 'ection, noncompliance, and review of 
their pupils* performance data on several programs. The cost of the 25 follow-up 
contacts was $250.77. 
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Application of Procedures 



The six teachers applied the procedures with 17 pupils in S3 programs for a total 
of 71 weeks (36% of the pupils served by these teachers). One teacher reported three 
times, two teachers each reported twice, and three teachers eoch completed one 
Procedures Questionnaire. 

Five of the teachers reported that the/ preferred collecting rate data: these 
teachers applied the procedures in programs in which they collected rate data 
exclusively. The teacher who had preferred to use percentage correct data also 
collected rate data on the one experimental program. All of the teachers used the 
rule procedure with minimum 'celerotion tines. 

Prior to the application of the procedures, five of the six subjects had rated the 
Handbook . Of these subjects, three rated the Handbook a median of 5.0. Two of these 
subjects gave the Handbook a median rating of 4*0 during oppltcation, while one 
continued to rate the handbook at 5.0. Two subjects rated the Handbook at 4.0 
overall, both before and Airing the application (Figure 3-1). The subject who did not 
rate the Handbook prior to application initially rated the Handbook at 4.0, then 
increased the median rating to 5.0 on the second reporting period (Figure 3-1). 
Ratings for each topic were generally lower during the application than prior to the 
actual use of the Handbook (Table 3-7). 
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Table 3-7 

SUBJECTS' RATINGS OF THE USEFULNESS 
OF Tf-E HANDBOOK DURING APPLICAITON OF THE PROCEDURES 



TOPIC 



RATING 
mean range 



I* How to Make cn Instructional Decision 

2. Quick Reference Guides for Change 
Strategies 

3. Drawing Lines of Progress 

4. Determining Percent Correct 

5. Determining Performance Variability 

6. Sharp Deceleration in Corrects 

7. General Considerations for 
Instructional Formats 

8. Acquisition 

9. Fluency-Building 

10. Compliance 

11. Overall 



4.50 4-5 



4. 50 
3.50 
3.33 
3.67 
4. 67 

4. 50 
4.17 
4.17 
4.67 
4.25 



4-5 

2- 5 
1-5 

3- 5 

4- 5 

4-5 
4-5 
4-5 
4-5 



Prior to ihe use of the experimental rules, t fc j six subjects reported planning for 
a median of four hours per week (range 2-10). During the application of procedures, 
the median was I hour per week (range 5-6). All of the teachers spent less time 
planning than before, although five teachers estimated that they spent more time. 
Figure 3-2 shows the change in reported planning time. 



Two of the six subjects rated their attitude to the procedures as "4" prior to 
application, during the use of the procedures, their ratings increased to M 5 M (Figure 3- 
3). Four subjects initially rated their attitude at "4", and only one of these subjects 
ERJC increased the attitude rating to "5" during application of the procedures. 
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FIGURE 3-1 

HANDBOOK RATINGS BEFORE AUD DURING APPLICATIOI1 OF DECISION RULES 
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FIGUR£ 3-2 



HOURS PLAHNING BEFORE AilD DURING APPLICATION OF DECISION RULES 
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Insert Figure 3-3 Here 

Impoct on Pupil Performonce 

The stx subjects applied the experimental procedures in o total or 53 program* 
Involving 1 7 pupils. Each subject shared some pupil performonce data, on a variety of 
target behaviors (Table 3-8). 

Subject 5402 applied the procedures to one program for one pupil for a total of 
56 calendar days and 18 data days (Table 3-9). During this time, the pupil did not 
progress to a new curricular step (step ahead). The teacher made one intervention, but 
did not follow the suggested strategy and changed late. The Jntervention was not 
successful, as would be predicted (Table 3*10). Subject 5402 estimated that the use of 
the experimental procedures made no difference to this pupil's progress. 

Subject 5403 shored data showing the application of the experimental procedures 
in three programs for one pupil. The charted pupil performance data showed no 
evidence that this subject drew new minimum toleration lines following the program 
changes; or evidence that Hnes-of~progress were drawn in order to use the flow chart 
to determine recommended strategies. Data for three of the programs included pupil 
performance prior to the application of procedures as well as during application. Prior 
to the use of the procedures, the pupil had moved ahead in the curriculum once in 1 1 
data days in a sorting program, and not at all in a 25 data day string beads program or 
a 10 day object identification program (Table 3-10). No other interventions were mode 
in these programs. During the use of the procedures, the subject stepped ahead a total 
of six times, and a total of eight other interventions were nnode (Table 3-10). When 
the subject followed the rules, these interventions were successful; when the rules 
were not followed, the interventions were not successful. The rules correctly 
predicted the outcomes of the interventions 88% of the time (Table 3-1 1). 

Subject 5404 applied the procedures in four programs with two pupils. These two 
subjects fell below minimum tolerations lines 30 times during the 237 days of data 
collected. Subject 5404 changed on time nine times, changed 2-4 days late on 20 
times, and did not change one time. After changing instructional procedures, 5404 
sometimes followed the procedures and drew new minimum T celeration lines, and 
sometimes continued using the line from the previous phase (not according to the 
procedures), tn the It strategy interventions that could he cnalyzed, the subject used 
the suggested strategy six times (Table 3-9). Overall, 58% of the interventions were 
successful, and the siftject used the suggested interventions 55% of the time (Table 3- 
1 1). The rules accurately predicted the outcomes of the interventions in 82% of the 
cases (Table 3-11). Subject 5404 reported that the procedures accelerated pupil 
progress in some programs and decelerated progress tn others. There are no data to 
indicate pjpil progress prior to the use of the procedures. 

Subject 5405 applied the procedures in 1 2 programs with five pupils. These five 
subjects fell below minimum toleration lines 27 times during the 540 days of data 
collected. Subject 5405 changed on time 16 times, eight times late ond did not change 
strategies on two occasions. Following the changes, 5405 did not always draw new 
minimum toleration lines as recommended by the procedures; if a new line were 
drawn, however, it was drawn from an incorrect starting point. There was no evidence 
that the subject drew It nes-of -progress on the charted data tn order to use the flow 
chart. Of the 29 strategy interventions mode by this subject, 27 were arwlyzable 
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FIGURE 3-3 

ATTITUDE TOWARD PROCEDURES BEFORE AM DURING APPLICATION OF DECISION RULES 



VERY 
FAVORABLE 



VERY 
UNFAVORABLE 



BEFORE 

5 



DURING 



. L.i — 

SUBJECT 5*102 



BEFORE 

5 



1 - 



DURING 



SUBJECT 5^*03 



BEFORE 

5 



DURING 



SUBJECT 5*05 



"VERY 
FAVORABLE 



"VERY 

UNFAVORABLE 



ER?C 



BEFORE 



DURING 



r 



SUBJECT 5*05 



BEFORE 
5 . 



DURING 



1 U 

SUBJECT 5*13 



BEFORE ' DURING 



l - 



SUBJECT 5*1* 

239 



ERLC 



£5 

(Tablt 3-9). Of these, 15 (56%) were successful. The subject used the suggested 
strategies 56% of the time, end the rules correctly predicted the outcomes of the 
Interventions used 92% of the time (Tdble 3-1 1) # Subject 5405 estimoted that the use 
of the procedures accelerated pupil progress, although there were no dato to indicote 
progress prior to application of the procedures. 

Subject 5413 applied the procedures in three programs *or one pupil. This 
subject used the procedures correctly, and collected a total of 62 days of dota in o 
totol of 142 colendar days (Table 3*9). During this time, the pupil moved ahead in the 
curriculum o total of 3 times. There were no other changes in curriculum level or 
instructional strategies, so the use of the rule procedures cannot be evaluated. 
Subject 5413 estimated that the procedures made no difference in pupil progress. 
There were no data to indicote pupil progress prior to application of the procedures. 

Subject 5414 opplied the procedures in II programs for three pupils. This subject 
olso toucfit the aide to utilize the procedures in 14 other programs for three other 
pupils. Both Subject 5414 and the oide opplied the procedures correctly. One of 5414^ 
pupils' dato included 192 days of data collected in three programs. During this time, 
5414 made o tatol of 24 interventions for compliance problems, only two of which 
were successful (8%). (Because of the unique nature and difficulties with compliance 
problems, the results for this pupil are presented separately in Toble 3-9). For the 
other two pupils, 14 of the 23 usable interventions were successful in changing pupil 
progress (61%). The teacher fallowed the suggested interventions 41% of the time, 
and the rules correctly predicted the outcomes in 82% of the cases (Toble 3-11). Doto 
showing pupil progress prior to the use of the procedures was mode ovoiloble (Toble 3- 
10). Generally, the pupils moved through the curriculum faster and the teacher mode 
mare interventions than before the use of the rule procedure. The teachers rate of 
successful interventions was higher when the suggested strategies were fallowed 
(Toble 3-10). Subject 5414 estimated that the use of the procedures occelerated pupil 
progress. 

Subject 5414*5 instructional oide was taught the procedures by 5414 The oide 
applied the procedures in 14 programs far three pupils over a total of 81 1 days of data 
(Toble 3-7). Most of the changes were step aheads, as the pupils moved through the 
curriculum (Toble 3*9). Of the 1 0 usable interventions, nine were successful. The oide 
followed the recommended strategy changes 90% of the time, and the rules were 80% 
successful in predicting the outcomes of those interventions (Toble 3-1 ih In mast 
cases, pupils progressed thrugh the curriculum faster (fewer days in each curriculor 
step) and intervention strategies were mare successful during the opplicotion of the 
procedures os compared to performance prior to application (Toble 3-10), 

Summory * As a group, these subject provided substontiol amounts of pupii 
performance dato collected during the application of the rule procedures (Tobie 3-9). 
However, except far the oide*s data, the percentage of successful interventions was 
less thon 61%; and the average, including the oide's 90% was only 59%. Further 
analysis shows, however, that subjects only chose to use the suggested interventions 
about 55% of the time (Toble 3-1 1). In other instances, teachers used their own ideas 
about what interventions would be successful. The use of o suggested strategy is o 
prediction thjt the pupil's progress will foil to occelerate. In the case of these 
subjects, predictions based on the rules proved accurate 85% of the time (Toble 3-12). 
There were 40 instances in which the subjects followed the rules, and progress wos 
accelerated in 35 of them (Toble 3-12). The subjects did not follow the suggested 
strategies in 33 cases, and, in 27 coses, the pupil's progress foiled to accelerate (Table 
3-1 2). 
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Data on 18 programs showed pupil progress both before and during application of 
the experimental rule procedures. One measure of progress ts how often the pupil 
moves to the next auricular step. Eoch program was designed by the teocher ; and the 
number of curricular steps, and the length of time It takes to go through those steps is 
partially determined by the task analysis as well as by the effectiveness of teacher 
programming* In seven programs, pupils moved ahead faster (shorter number of days 
to reoch aim) during the period when the rules were applied (Table 3-10) than they did 
when the rules were not in effect. On the other hand, pcpils in six programs moved 
ahead faster prior to rule application. Thus, the rules, as applied by these subjects, 
may not have affected pupil progress, given that recommended interventions were 
generally made 55% of the time. 

Teachers generally did make more interventions during the rule application 
(Table 3-10) than before tne rule application, and the percentage of successful 
interventions was generally higher, especially when the teacher used one of the 
recommended strategies. 

Cost Analysis 

The total costs for training 1 1 subjects at this site are shown in Table 3-13. Of 
the 1 1 trained subjects, six actually utilized the procedures In programs for 17 pupils 
over a total of 71 weeks. If ali site costs are included, these figures result in an 
overage per pupil cost of $38.34 for those pupils for whom the experimental 
procedures were applied, or approximately $9.18 per week of application for each 
teacher. 

If costs are separated to include only those costs incurred in the training subjects 
who acutalty used the procedures, these figures are reduced to a median cost of $15.1 5 
per pupil and $4.78 per week (Table 3-IW. For subjects who applied ,he procedures, 
the cost per program per week averaged $.38. Figures for the subjects vary 
consider obi y. 
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Table 3-8 

TARGETED MOVEMENT CYCLES IN PUPIL PERFORMANCE DATA 
Age Rcnge of Pupils Movement Cycles* 

1-6 touches table 

6-12 identifies objects; strings beads; sort* objects; names currency; names numbers; reads words 

orally; says sight words; writes digits to answer odd facts; points to named currency; 
maintains eye contoce; reoch cjid touch; folds, staples cjid stuffs; follow directions; answers 
questions about reading; reads sentence and matches to picture. 

8-17 points to named colors; snaps pants; says phone number; sjy& name and address* 

12-22 writes name, address cjid phone number; files alphabetically; says time of day; assembles 

booklets; touches named clothing; walks balance beam; sorts objects; copies block design; 
uses communication boards types letters; folds, staples cjid stuffs; collates pages; touches 
cards '^eat w "milk"; says tetters; articulates sounds; says emergency words; identifies money; 
speaks intelligibly; uses language cards; writes name. 

* Movement cycles are given generally as described by subjects; most subjects counted both correct aid error responses 
although that distinction is not mode above 
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Table 3-9 

ANALYSIS OF PUPIL PERFORMANCE DATA 
Application of the Minimum 'Celerotton Procedures 



Subject 


Pupils 


Programs 


Calendar 
Day* 


Data 
Don 


Phase* 


Step 
Abends 


Inter, 
vent for* 


Usable 


# Sheets 
Inhtr. 


% Success 
hterv. 


Subject's Estimate of 
Impact of Proce-turej 


5402 


1 


1 




18 


2 


0 


1 


1 


0 


0% 


no difference 


5*03 


1 


3 


267 


138 


17 


6 


8 


6 


2 


33% 


no difference In tome 
accelerated some 


5404 


2 


4 


387 


237 


26 


3 


20 


12 


7 


58% 


both accelerated and 
decelerated 


5405 


5 


12 


1220 


540 


68 


45 


29 


27 


15 


56% 


accelerated 


S413 


1 


3 


142 


62 


10 


8 


0 


0 






no difference 


S4I4* 


2 


8 


1136 


5t3 


97 


4! 




23 


14 


61% 


accelerated 


S4I4 

ofcfe 


3 


14 


1752 


811 


96 


66 


20 


10 


9 


90% 


accelerated 


TOTAL 


15 


4S 


4960 


2319 


31$ 


169 


126 


79 


47 


S9% 





• in oddition, 1 92 days of data were collected over 504 calendar days in three programs for one other pupil. In 35 phases, the pupil met 
aim and moved to the next curricula- step (step ahead) 7 times. Twenty-four Interventions were made for compliance problems} of the 
22 usuoble interventions, 2 (9%) were effective. 
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Table 3*10 




INDIVIDUAL PUPIL PROGRESS BEFORE AND DURING APPLICATION OF EXPERIMENTAL PROCEDURES 



Before Uje of the Rules During U** of Rules 



Subject 


Pupil 
No. 


Behavior 


Dote 
Days 


Step A v. Day* 
Aheods To Step 
Ahead 


Inter- 
ventions 


* Success 
Interven- 
tions 


Data Step 
Dors Aheods 


A v. Days 
To Step 
Ahead 


Inter - 
vent* 
tans 


% Success 
Inter ven- 
ions 


% Follow 
Rules 


5403 




identify 
obj. 


10 


0 


*> 


0 


*> 


43 


1 


43 




0% 


0% 


5403 




tiring 
beads 


25 


0 


*> 


0 


*> 


47 


3 


14 




100% 


100% 


5403 




sorts 

obj-, 


Jl 


1 


II 


0 




48 


2 


24 


2 


100% 


100% 


5414 




oss em* 
bles 


21 


I 


21 


2 




72 


10 


3 




100% 


100% 


54H 




touch 
clothing 


20 


o 


•> 


3 


0% 


Kt 


J 


17 


6 


66% 


33% 


5414 




walks 

Kn ! K*Arn 


16 


3 


s 


0 




£1 
OJ 


7 


9 


5 


75% 


75% 


54U 


1 




52 


3 


17 


8 


33% 


65 




16 


6 


33% 


33% 


5414 


2 


copies 

block 

desig/i 


10 


0 


*> 


( 


100% 


89 


13 


7 


7 


90% 


80% 


2 


pts to 

long. 

board 


29 


0 


*> 


2 


0% 


8J 


3 


27 


< 


100% 


50% 


5*14 


2 


t^Pes 
letters 


20 


1 


10 


0 


7 


35 


t 


35 


2 


0% 


0% 


5*1* 
old* 


1 


ident* 
money 


45 




22 


1 


100% 


68 


8 


8 


,J 


100% 


100% 


5*1* 
oide 


I 


soys 
letters 


56 




14 


1 


100% 


72 


6 


12 


0 


*> 


7 


5*14 
oide 


i 


SOYS 

cm era 
si 9^5 


49 


2 


24 


1 


0% 


St 


4 


13 


0 


*> 


7 


5*1* 
aide 


2 


soys 
em erg. 
signs 


50 




17 


2 


50% 


47 


4 


12 


2 


50% 


50% 


5*14 
aide 


2 


toys 
letters 


13 




6 


1 


7 


69 


3 


23 


1 


100% 


100% 


5*1* 
oide 


2 


files 

olphabeT. 
}co\\y 


50 




8 


2 


50% 


S3 


5 


11 


2 


100% 


100% 


5*14 
oide 


3 


pts, to 
Janj. cord 


3: 




S 


0 


7 


64 


7 


9 


0 


7 


7 


Wl. 


"j 


IdenK 
money 


42 




2i 


2 


50% 


60 


3 


20 


2 


J 00% 


100% 
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Table 3-H 

UTILIZATION OF SUGGESTED STRATEGIES 



Subject 


# Interventions 


ff Usable 


0 Successful 


% Successful 


0 Using Suggested % Using Suggested 
Strategy Strategy 


Predictability 
of the Rules 


5402 


1 


1 


0 


0% 


0 


0% 


100% 


5403 


8 


6 


2 


33% 


3 


50% 


88% 


5404 


20 


12 


7 


58% 


6 




82% 


5405 


29 


27 


15 


56% 


15 


56% 


92% 


54)3 


0 














5414* 


48 


23 


14 


61% 


7 


41* 


82% 


54(4 
aide 


20 


10 


9 


90% 


9 


90% 


80% 


TOTALS 


126 


79 


47 


59% 


40 


55% 


85% 



* of 24 interventions mode for another pupil, all were for compliance problems. Of the 24, 22 were analyzable, and 16 were Inoccord with the 
suggested change strategies; I of these was successful. The compliance rule accurately predicted the results of the interventions 27%* 
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Table 3-12 



ACCURACY OF RULES FOR INTERVENTION CHANGES 



Pupil Progress Pupis Progress 

Actually Accelerated Did Not Actually Accelerate 

Change mode in occord with a rule 48%* 7% 

(Prediction that progress will (N=35) (Nr5) 

be occelerated) 



Change mode not in occord with rule 8% 37%* 

(Prediction that progress will not (N=6) (Nr27) 

be occelerated) 

* Concurrence of prediction ond octual results indicates rule accuracy: total 62/73 85% 



Table 3-13 
COST SUMMARY 



ALL COSTS 

No. Subjects Trained : 1 1 

Total Handbook Costs : $386.65 

Total Follow-up Cost : 5 265. 1 1 

TOTAL SITE COST : § 651.76 

Av. Per Subject Cost : $ 59.25 

Av. Per Pupil Cost* : $ 38.34 

A v. Per Week Cost** : $ 9.18 



* Includes N= 1 7 pupils whose teachers applied the procedures in instructional programs 
** Includes N=7I weeks during which teachers applied procedures 
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Table 3-14 

COSTS FOR SUBJECTS APPLYING PROCEDURES 



Application of^rocedures AM Costs Cost Analysis 



Subject 


Weeks 


Pupils 


Pro 'arm 


Training 


Follow-up 




Total 


Per Pupil 


Per Program 


Per Week 


Per 

Program/Week * 


5402 


8 


1 


1 


$ 


35.15 


$ 


40.70 


$ 


75.85 


$ 75.85 


$ 75.85 


$ 9.48 


$ 9.48 




& 


I 


3 

J 


$ 


35.15 


$ 


60.49 


$ 


95.04 


y 7 J • OH 


< it on 

y J 1 . 00 


< |C OA 

9 1 J • W 


ten 


5404 


16 


3 


6 


$ 


35.15 


$ 


29.08 


$ 


64.23 


$ 21.4! 


$ 10.71 


$ 4.01 


$ .70 


5405 


16 


5 


12 


$ 


35.15 


$ 


37.48 


$ 


72.63 


$ 14.53 


$ 6.05 


$ $.54 


$ .38 


5413 


6 


1 


4 


$ 


35.15 


$ 


25.24 


$ 


60.39 


$ 60.39 


$ 15.10 


$ 10.07 


$ 2.52 


5414 


19 


6 


2( 


$ 


35.15 


$ 


55.72 


$ 


90.87 


$ 15.15 


$ 3.50 


$ 4.78 


$ .18 


TOTAL 
MEDIANS 


71 


17 


53 


$ 210.90 


$ 248.71 


$ 459.61 
$ 74.24 


$ 15.15 


$ 6.05 


$ 4.78 


$* .38 
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SITE a 
Introduction 

Ti»£ cooperative arrangement between the Seattle Public Schools and the 
Instructional Hierarchies Research Project, Initiated during the fourth project year, 
was continued during the fifth project year following consultations with Dr. Bill Tilley, 
Director of Special Services. 

Seattle is a large urban city with a population af 497*300 as of April I, 1979. 
Mandatory busing for racial desegregation is presently in effect. There are 112 
schools within the Seattle Public School system, 65 schools serve both handicapped 
and nonhandtcopped students end three schools serve handicapped students exclusively. 
There are 47,662 nonhandicapped students enrolled in the district and 2,513 
handicapped students. 

Three schools* Green Lake Elementary and Lowell Elementary (serving 
handicapped and nonhandicapped pupils preschool - Grade 5), end Wilson Pacific High 
School (serving handicapped pupils aged 13-22) and Lowell Elementary (serving 
handicapped and nonhandicapped pupils, preschool - Grade 6) were selected for 
participation because the majority of classrooms serving severely, profoundly and/or 
muftiply handicopped pupils in the district were located in these schools. Meetings of 
approximately 30 minutes were held separately with eoch principal. During ejeh 
meeting, members of the research staff presented an overview of the purpose of the 
project and provided copies of the consent form (Appendix 4) and the Background 
Questionnaire. Each prospective subject met tnclvidually or in a small group with 
interested teachers, same of whom were recommended by th principal* Twelve 
teachers consented to participate. Subjects were trained both in small groups and in 
individual sessions* 

Method 

Subjects 

All 12 of the consenting teachers were experienced teachers (mean = 9.6 years 
teaching, range 3-22 years). The overage number of years teaching special education 
was 8.25, ranging from 3-22 years. Eight of the 12 teachers reported formal training 
in Precision Teoching. 

The 12 teachers served 215 pupils with the following "labels": learning disabled 
(8); mildly handicapped (14); moderately handicapped (68); severely handicapped (76); 
profoundly handicapped (32); multiply handicapped (I l), and arthapedicaliy 
hondicapped (6). Three of the teochers served a totoi of 135 pupils: one served 36 
pupils in a team teaching situation and two served 50 and 49 pupils respectively in 
resource rooms. The average class size of the other nine subjects was 8.4 pupils (range 
5-16). T*o teachers served a total of 41 preschool pupils (aged 2-6), five teachers 
served 3? elementary pupils (aged 6*12), one teacher served five mixed preschool and 
elementary pupils (aged 3-11) and four teachers served 127 high school pupils (aged 12- 
22). 

Each of the 12 teochers hod some assistance in the classroom. There were a 
totol of 23 paid assistants* five volunteers, 16 student teochers and nine other people* 
for 644 hours per week. The subject with the most assistance had a total of six people 
helping for 121 hours per week, while the subject who had the least had two people 
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assisting for eight hours per week. The overoge amount of assistance available to eoch 
of the 12 teachers 12 hours per week. Pupils in 1 1 of the 12 classooms received an 
average of 1 5 hours of therapy assistance weekly (range 1-40 hours). 

All of the teochers provided bath 1:1 and group settings for Instructional 
responding. Three teochers spent the majority of instruction in Ttl settings and two 
spent the majority of instructional time in group settings. The other teochers divide 
their time between those two settings, and using natural occasions to teach new 
responses. Three teachers also use unison responding a small portion of the time. Ten 
of the teochers conduct instructional progroms at least once per day, while two 
conduct programs three times per week. Six teachers conducted at least 10 trials per 
program, while three provided five-nine trials, and three provided fewer. 

All 12 teachers collect some dcta on student performance. Three teachers 
co'lect data on 95-100% of the programs, four teachers collect data on 85-94% of the 
programs, one teacher on 75-84%, two teochers on 25-49% and two teochers collect 
data on 10-24% of the programs. Six teachers collect data every time an instructional 
program is conducted, and six teochers collect data almost every time a targeted 
program is conducted. Six subjects collected data on every trial of their data-based 
programs. Two subjects collected data on "probe* 1 trials only, one on the first trial 
only, one on a random number of trials, one on a consistent number of trials as stated 
In the program and one teacher collects data on every trial on a new program and on 
probe trials in established programs. The most frequently used data type was accuracy 
data, although use of all other data types was reported. 

Three teachers graphed their data. All have definite rules in at least some of 
the programs for deciding when to step ahead and when the pupil has met aim. Seven 
teochers reported hoving decision -rules for at least some of their programs to 
determine when to change instructional procedures rather than the program. On|v one 
subject reported specific rules for deciding how to modify instructional procedures. 

Training 

The training sessions were scheduled at the subjects convenience, either before 
school, during the subjects regularly scheduled planning time, or lunch time. There 
were generally four or five sessions each lasting about 30 minutes. Six subjects were 
trained individually and six subjects were trained in small groups of two and four 
teachers. Total training t:me ranged from 1.9? hours to 3.38 hours (Table 4-1). 

The content of the training sessions was identical for all subjects. Prerequisites 
for use of the data decision -rules and a general introduction to data and data 
collection were covered during the first training session. Subjects received a handout 
covering these topics, prepared for the October AAESPH conference in Chicago. The 
second and sometimes third training session covered data collection and charting of 
regular rate, adjusted rate, latency, and duration (trial and session) data. The subjects 
practiced collecting and charting eoch type of data. At the next session, subjects 
were tou$Jht how to set a minimum toleration line, draw I ines-of -progress and apply 
the decision rules. The final training session covered using the Handbook, and 
acquisition, fluency-building and compliance strategies. At the end of this session 
subjects were asked to complete a Presentation Questionnaire and given the Handbook 
of Experimental Proc e dures . 
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n 



Subject 
1:1 Training 
5501 
5502 
5503 
5504 
5507 
5508 

Median 



Table 4-1 
Length and Cast of Training 
Length of Training Cost 



2. 5 hours 
3,01 hours 
3.38 hours 
3.08 hours 
I .92 hours 
2.25 hours 

2.75 hou.. 



$62. 40 
$73. 49 
$81.07 
$75.02 
$49.31 
$56.98 

$67.95 Total 



$398.77 



1 :2 Training 
5513 
5514 



2.67 hours 
2.67 hours 



$63.16 

$63.16 Total = $126.32 



1:4 Training* 
5509 
5510 
5511 
5512 



2.34 hours * 
2.34 hours 
2.34 hours * 
2.34 hours 



$43.91 
$41.47 
$45.42 

$41.47 Total 



$172.27 



* includes individual make-up sesssion 



TOTAL $697.36 



Follow-up After Training 

Two subjects requested some additional information following training. This 
information was provided in three pSone conversations for a total cost of $2.45 and a 
visit of $4.32. Total cost, excluding mileage, wc .i $6.14. 
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Evgluotjon of Handboo k 

Eoch of the 12 teachers received a copy of the Handbook of Experimental 
Procedures , and two weeks later eoch received the Handbook Questionnaire. Eleven 
subjects completed the questionnaire white one subject reported insufficient time +o 
read the htandbookt Teachers spent on average of two hours and five minuses 
reviewing the Handbook (range 10 minutes to Sfc hours). They rated the section on 
compliance as the clearest (4.5) ~ 1 the Information on rules for fate and the Quick 
Reference Guides as the leost ciear (Table 4*2), and the median rating was 4.2, as 
before application. 

Ten teachers wrote comments on the Hqndbook . They suggested a Table of 
Contents and additional examples of programs for low-functioning pupils be added 

Table 4-2 

SUBJECTS' RATING OF CLARITY OF INFORMATION PRESENTED IN THE 
HANDBOOK OF EXPERIMENTAL PROCEDURES 

Content Area Ratings <N= 17) 

mean (range) 



Using data decision-rules 


3.8 


(2-5) 


Rules for rate and quick reference guide for rule* for rate 


3.6 


(2-5) 


Quick reference guides for change strategies 


a. 3 


(2-5) 


Drawing li nes-of -progress 


3.8 


(2-5) 


General considerations for Instructional formats 


it. 2 


(2-5) 


Acquisition 


it.2 


(2-5) 


FUiency-building 


it.5 


(2-5) 


Compliance 


it.2 


(2-5) 


Median Rating 


it.2 


(2-5) 



Teachers <N=I I) 

Adoption of Procedures . Although all 12 subjects ogreed to apply the procedures 
to some programs] only ' I returned Procedures Questionnaires indicating that they 
actually appfftd the procedures. These subjects served a tr'al of 133 pupils per year 
(Tobfe ft-3). 
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Table 4-3 
APPLICATION OF PROCEDURES 



Subject 


Pupils 


Handicapping Conditions 


Age Range 


Application of Procedures 
Pupils Programs Weeks 


5501 


34 


learning, disabled, mild, 
moderaTe, severe, profound 


3-6 


1 


1 


12 


5502 


6 


moderate, severe 


6-10 


6 


6 


19 


5503 


9 


moderate 


8-12 


2 


5 


5 


5504 


6 


profound 


7-1 1 


1 


1 


12 


5507 


6 


severe, profound 


3-12 


2 


it 


£ 

U 


5508 


5 


profound 


12-21 


1 


1 


6 


5510 


15 


severe 


13-21 


I 


1 


5 


5511 


13 


severe 


13-21 


* 

i 


1 


5 


5512 


27 


severe, profound 


13-21 


2 


2 


5 


5513 


6 


severe, profound 


7-12 


2 


4 


4 


5514 


6 


severe, profound 


9-12 


3 


5 


6 



TOTAL 133 22 31 97 
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Additional Follow-up 

Sixteen follow-up requests were received. The topics covered during the follow* 
up visits included: charting (6 subjects); data collection (2 subjects); drawing minimum 
V^eleration lines (2 subjects); setting aims (I S'fcject); drawing lines-oNprottress (I 
subject) and interpreting pupil data ( I subject). The cost of the IS follow-up visits was 
$1 15.72, (see Table 4-10 for per subject costs). 

Application of Procedures 

The 1 1 teachers applied the procedures with 22 pupils in 31 programs for a total 
of 97 weeks. The 22 pupils represent cbout 17% of the pupils served by these teachers 
(Table 4-3). One teacher report three times on the application of the procedures, 
four teachers reported twice, and six teachers reported once each. 

Prior to application of the experimental procedures, six teachers reported that 
they collected percent correct data most often, one collected counts of trials, one 
collected counts of behavior and two collected rate data most often. However, during 
the experimental procedures, time based data (rate, duration or latency) were 
collected in all but one of the programs. All of the teachers used the rute procedure 
with minimum feeleration lines* 

Many of the subjects rated the Handbook content lower during the application of 
the procedures than prior to its use (Figure 4-1). In general, the technical content 
areas tended to be rated as before, while the descriptive sections (e.g., "acquisition") 
dropped in ratings (Table 4*4). 



INSERT FIGURE 4-1 HERE 



figure 4-1 
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Tdble 4-4 

Rotir>g of Handbook Content During Application of Procedures 



Content Area Ranking Before Rartking During 

Use Application* 

N=l| N=I0 

How to make an instructional decision 3.82 4.6 

Rules and quick reference guides 3.64 4*6 

Technical procedures (draw minimum 'celeration 3*8 3.6 
lines; draw lines of progress; determine 
hihg variability; determine percent correct) 

Designing instructional format 4.2 3.8 

Acquisition h.2 3.2 

Fluency Building 4.2 3.7 

Compliance 4.5 4.3 

Overall i.2 3.9 



# Includes only reports during first Procedures Quest tonnai re; for changes in ratings over 
the application period, see Figure 4-'* 



Prior to the application of the experimental procedures, the I ! subjects reported 
spending a median of four hours oer week planning (range 2-5J. During the application 
of procedures, the medial was I Sour per week (range ,5-2). All of the teachers spent 
less time planning instruction than before, although five of them estimated that they 
spent more time, and rour estimated that they spent the same amount of time planning 
as previously. Figure 4-2 shows the change in reported plannir time* 



INSERT FIGURE 4-2 HERE 



The attitude of four of the teachers toward the procedures dropped du r ing acttol 
application (Figure 4-3), while the attitude of seven teachers remained the same* The 
change in attitude is reflected in the means shown in Table 4-5, and cut across all 
aspects of the procedures except collec+ing data. 
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FIGURE 4-2 

HOURS PLANNING BEFORE AND DURING APPLICATION OF DCCISIO.4 RULES 
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Table 4-5 

* 

Average Rating of Attitudes Toward Procedures 



Procedures 


Attitude Prior 
To Application 


Attitude During 
Application 


Establish instructional programs 


4.75 


4.1 


Collect data on pupil performance 


4.8 


4.8 


Chart pupil data 


4.5 


3.7 


Use experimental rules 


4.8 


3.9 


Use suggested strategies 


4.75 


3.8 


Overall 


4.6 


4.2 



Impact on Pupil Performance 

Five subjects shored data of eight pupils in 12 programs on a variety of behaviors 
(Table 4-6). Six of the 1 1 subjects indicated that they thought that the application of 
the experimental procedures hod accelerated pupil progress* Four of these teachers 
provided some pupil performance data (Table 4-7)* Subject 5504 made two changes in 
the one experimental program conducted, and both of these were step aheads* Thus, it 
is impossible to evaluate the Impact of the rules in this particular program* Subject 

5502 mode only one intervention, and that was not in accord with the rules, and the 
Intervention did not improve pupil performance (Table 4-7 ond Table 4-8)* However, 
this subject applied the procedures in four otl*r programs not submitted* Subjects 

5503 and 5508 each hod three usable interventions in the dato submitted to the 
project, and the majority of these interventions were successful* Thus, of the four 
subjects who estimated that pupil performance was accelerated, and whom submitted 
pupil performance data, twc show some Jevei of successful intervention firing the use 
of the procedures* Since data indicating pupil progress pri^ to application of the 
procedures are not available, no comparison with previous progress can be made* AM 
of these subjects plan on using the procedures in the future* 

Four of the 11 subjects indicated that the procedures made no difference to pupil 
performance* One of these sub; *cts submitted program data (Table 4-7)* Subject 551 2 
collected 41 days of data on t a programs with two pupils* During this time no 
changes or Interventions of any k id were made* Since no changes or interventions 
were made it Is difficult to e&'mate the impact of the rules on the pupil's 
performance* AH of these subjects plan on using the procedures in the future* 

Subject 5501 reported twice, at first indicating that pupil progress accelerated 
during the use of the procedures, ond then that the procedures did not offect pupil 
progress* This subject did not provide any pupil pertormonce data, but plans on 
continuing to use the procedures* 
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Cost Analysis 

The total costs for troining 12 subjects at this site are shown in Table 4-9. Of 
the 12 subjects, 1 1 aetuoJy utilized the procedures in programs for 22 pupils over o 
total of 97 weeks, (f all site costs are included, these figures result in an average per 
pupil cost of $37.26 for those pupils for whom the experimental procedures were 
applied, or approximately $8.45 per week of appTcttion per teacher. 

If costs are separated to iclude only those costs incurred in training subjects who 
actually used the procedures, these figures ere reduced to a median cost of $23.45 per 
pupil and $6.61 per week per teacher (Table 4-10). If costs are cotculoted per progrom 
per week, the median cost Is $4.69. 



Table 4-6 



Age Range 



Movement Cycles 



3-4 



holds head up 



6-12 



spells words, pulls loop tight tshoes), traces upper case 
letters, reads words orally, points to letters of name, 
touches object 



13-21 



assembles sifters, assembes gas valves 



2tK) 




Table 4-7 

ANALYSIS OF PUPIL PERFORMANCE DATA 
Application of the Minimum 'Celerat!on Procedure 



Subject 


Pupils 


Programs 


Calendar 
Days 


Data 
Days 


Phases 


Step 
A heads 


Inter- 


Usable 


#Success- 
ful 


% Success- 
ful 


Subject's 

P*t1mfrte f%f 

Impact of 
Procedures 


5502 


2 


2 


134 


28 


5 


2 


1 


1 


0 


0% 


Accelerated 


5503 


2 


> 
6 


288 


120 


23 


13 


4 


3 


3 


100% 


Accelerated 


5504 


1 


1 


29 


20 


3 


2 


0 








Accelerated 


5508 


1 


1 


122 


72 


5 


1 


3 


3 


2 


66% 


Accelerated 


5512 


2 


2 


72 


41 


2 


0 


0 








Accelerated 


TOTAL 


8 


12 


645 


281 


38 


!8 


8 


7 


5 


71% 
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Table 4-8 

UTILIZATION OF SUGGESTED STRATEGIES 

Subject # Interventions 0 Usable P Successful % Successful # Using Suggested % Using Suggested Predictability 

Strategy Strategy of Rules 

5502 I I 0 0% 0 0 100% 

5503 4 3 3 100% 2 66% 66% 

5504 0 .... 

5506 3 3 2 66% 2 66% 66% 

5512 0 

TOTAL 8 7 5 71% 4 57% 71% 
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Table 4-9 
COST SUMMARY 

No. Subjects Trained t 12 

Total Training Cost: $697.36 

Total Follow-up Cost: $121 .86 

TOTAL SITE COST: $819.22 

Average Per Subject Cost: $ 68.26 

Average Per Pupil Cost: $ 37.26 

Average Per Week Cost**: £ 8.45 

Includes N=22 pupils whose teachers applied the procedures 
Includes N=97 weeks during which teaches applied procedures 
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Application of Procedure s 



Table MO 

COSTS FOR SUBJECTS APPLYING PROCEDURES 
All Costs 



Cost Analysis 



Subject 


Weeks 


Pupils 


Programs 


Training 


Follow-up 




Total 


Per Pupil 


Per Program 


Per Week 


Per 

Program/Week 


5501 


12 


1 


I 


$ 


£2.40 


$ 


6.14 


$ 


68.54 


$ 68.54 


$ 68.54 


$ 5.71 


$ 5.72 


5502 


19 


6 


6 


$ 


73.49 


$ 


52.17 


$ 


125.66 


$ 20.94 


$ 20.94 


$ <S.6l 


$ t.to 


5503 


16 


2 


5 


$ 


81.07 


$ 


14.34 


$ 


95.41 


$ 47.71 


$ 19 08 


$ 5.96 


$ 1.19 


5504 


12 


1 


I 


$ 


75.02 


$ 


0.00 


$ 


75.02 


$ 75.02 


$ 75.02 


$ 6.25 


$ 6.27 


5507 


6 


2 


4 


$ 


49.81 


$ 


0.00 


$ 


49.81 


$ ?^.9t 


$ 12.45 


$ 8.18 


$ 2.08 


5508 


6 


1 


t 


$ 


56.98 


$ 


9.35 


< 

V 


66.33 


$ 66.33 


$ 66.33 


$ tl 


$1 ! .06 


5510 


6 


1 


I 


$ 


41.47 


$ 


25.18 


$ 


66.65 


$ 66.65 


$ 66.65 


$ It. It 


$11.11 


551! 


5 


t 


1 


$ 


45.42 


$ 


3.63 


T 


49.11 


$ 49.11 


$ 49.11 


$ 9.82 


$ 9.82 


5512 


5 


2 


2 


$ 


41.47 


$ 


5.43 


$ 


46.90 


$ 23.45 


$ 23.45 


$ 9.98 


$ 4.69 


5513 


4 


2 


4 


$ 


63. 16 


$ 


0.00 


$ 


63.16 


$ 31.58 


$ 15.79 


$ 15.79 


$ 3.95 


5514 


6 


3 


5 


$ 


63.16 


$ 


0.00 


5 


63.16 


$ 21.05 


$ 12.63 


$ 10.53 


$ 2.11 


TOTAL 
MEDIANS 


97 


22 


31 


$653.45 




$116.24 




$769.69 
$68.54 


$23.45 


$19.08 


$6.61 


$4.69 
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